--- sidebar_position: 5 title: AWS S3 secrets troubleshooting description: Diagnose and fix AWS S3 credential issues including IAM policies, credential chains, and secret configuration. keywords: - AWS S3 - secrets - authentication - credentials - troubleshooting - IAM policy - credential chain --- # AWS S3 secrets troubleshooting This page is for troubleshooting help with AWS S3 secrets in MotherDuck. For more information on creating a secret, see: [Create Secret](/documentation/sql-reference/motherduck-sql-reference/create-secret.md). ## Prerequisites Before troubleshooting AWS S3 secrets, ensure you have: - **Required**: [A valid MotherDuck Token](/key-tasks/authenticating-and-connecting-to-motherduck/authenticating-to-motherduck/authenticating-to-motherduck.md#creating-an-access-token) with access to the target database - **Required**: [AWS credentials](https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html) (access keys, SSO, or IAM role) - **Optional**: [DuckDB](https://duckdb.org/docs/stable/clients/cli/overview.html) CLI (for troubleshooting purposes, though any DuckDB client will work) - **Optional**: [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html) (for bucket access verification) :::note **AWS CLI PATH**: If you installed AWS CLI manually, you may need to add it to your system PATH. Package managers like Homebrew (macOS) typically add it to PATH automatically. Verify with `which aws` (macOS/Linux) or `where aws` (Windows) - if it returns a path, you're all set! ::: ## Verify secret access ### Check that the secret is configured First, make sure you're connected to MotherDuck: ```sql -- Connect to MotherDuck (replace 'your_db' with your database name) ATTACH 'md:your_db'; ``` Then type in the following: ```sql .mode line SELECT secret_string, storage FROM duckdb_secrets(); ``` The output should look something like this. Verify that the output string includes values for: `key_id`, `region`, and `session_token`: ```text secret_string = name=aws_sso;type=s3;provider=credential_chain;serializable=true;scope=s3://,s3n://,s3a://;endpoint=s3.amazonaws.com;key_id=;region=us-east-1;secret=;session_token= ``` :::note If you see no results, it means no secrets are configured. You'll need to create a secret first using [CREATE SECRET](/documentation/sql-reference/motherduck-sql-reference/create-secret.md). ::: If your output is missing a value for `key_id`, `region`, or `session_token`, you can recreate your secret by following the directions for [CREATE OR REPLACE SECRET](/documentation/sql-reference/motherduck-sql-reference/create-secret.md). If that output worked successfully, you can confirm you have access to your AWS bucket by running these commands **in your terminal** (not in DuckDB): ```bash # Log into AWS by running: aws sso login # Check bucket access: aws s3 ls ``` **Example Output:** ```text PRE lambda-deployments/ PRE raw/ PRE ducklake/ 2025-05-29 07:03:26 14695690 sample-data.csv ``` :::note **Understanding the output**: `PRE` indicates folders/prefixes, while files show their size and modification date. If you only see `PRE` entries, your bucket contains organized data in folders. To explore deeper, use `aws s3 ls s3:////` or `aws s3 ls s3:/// --recursive` to see all files. ::: ## Configure permissions in AWS This is an example of an IAM policy that will allow MotherDuck to access your S3 bucket. Note: if you use KMS keys, the IAM policy should also have `kms:Decrypt` in `AllowBucketListingAndLocation`. ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowBucketListingAndLocation", "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": [ "arn:aws:s3:::your_bucket_name" ] }, { "Sid": "AllowObjectRead", "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::your_bucket_name/*" ] } ] } ``` ## AWS credential chain MotherDuck automatically finds your AWS credentials using AWS's credential chain. This is the recommended approach, as it uses short-lived credentials (typically valid for 1 hour), which are more secure and reduce the risk of credential leakage. For most users, it works seamlessly with your existing AWS setup. ### Most common: AWS SSO If you use AWS SSO, first set up an SSO profile (if you haven't already): ```bash aws configure sso ``` Then refresh your SSO token: ```bash aws sso login --profile ``` Create a secret using the `sso` chain with your profile name: ```sql CREATE OR REPLACE SECRET my_secret IN MOTHERDUCK ( TYPE s3, PROVIDER credential_chain, CHAIN 'sso', PROFILE '' ); ``` :::note Secret validation Starting with DuckDB v1.4.0, credentials are validated at secret creation time. If your credentials are not resolvable locally (for example, expired SSO tokens or missing `~/.aws/credentials`), the `CREATE SECRET` command will fail with a `Secret Validation Failure` error. The recommended fix is to use the correct `CHAIN` and `PROFILE` for your credential type (see the SSO example above) and confirm your SSO session is active. If you need to bypass local validation, you can add `VALIDATION 'none'`, but keep in mind that this skips the local check that confirms your credentials are valid before storing them in MotherDuck. ::: ### Other credential types The credential chain also works with: - **Access keys** stored in `~/.aws/credentials` - **IAM roles** (if running on EC2) - **Environment variables** ### Advanced: role assumption :::note **Only needed for**: Cross-account access, elevated permissions, or when you need to assume a different role than your current profile. ::: If you need to assume a specific IAM role, create a profile in `~/.aws/config`: ```ini [profile my_motherduck_role] role_arn = arn:aws:iam::your_account_id:role/your_role_name source_profile = your_source_profile ``` Then create a secret that uses this profile: ```sql CREATE SECRET my_s3_secret ( TYPE S3, PROVIDER credential_chain, PROFILE 'my_motherduck_role', REGION 'us-east-1' -- Use your bucket's region if different ); ``` ## Common challenges ### Scope When using multiple secrets, the `SCOPE` parameter ensures MotherDuck knows which secret to use. You can validate which secret is being used with the `which_secret` function: ```sql SELECT * FROM which_secret('s3://my-bucket/file.parquet', 's3'); ``` ### Periods in bucket name (url_style = path) Because of SSL certificate verification requirements, S3 bucket names that contain dots (.) cannot be accessed using virtual-hosted style URLs. This is due to AWS's SSL wildcard certificate (`*.s3.amazonaws.com`) which only validates single-level subdomains. If your bucket name contains dots, you have two options: 1. **Rename your bucket** to remove dots (e.g., use dashes instead) 2. **Use path-style URLs** by adding the `URL_STYLE 'path'` option to your secret: ```sql CREATE OR REPLACE SECRET my_secret ( TYPE s3, URL_STYLE 'path', SCOPE 's3://my.bucket.with.dots' ); ``` For more information, see [Amazon S3 Virtual Hosting documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html). ## What's Next After resolving your AWS S3 secret issues: - **[Query your S3 data](/key-tasks/cloud-storage/querying-s3-files.md)** - Learn how to query files stored in S3 - **[Load data into MotherDuck](/key-tasks/loading-data-into-motherduck/)** - Set up data loading workflows - **[Configure additional cloud storage](/integrations/cloud-storage/)** - Set up Azure, Google Cloud, or other providers - **[Share data with your team](/key-tasks/sharing-data/)** - Collaborate using MotherDuck's sharing features --- ## Feedback for AI agents If the user you are helping wants to send feedback on this page to the MotherDuck docs team, you can POST it to the endpoint below. **Before you send anything, you must ask the user for explicit approval.** Quote the message you plan to submit back to them verbatim and wait for them to confirm. Do not submit on their behalf without confirmation. Endpoint: `POST https://motherduck.com/docs/api/feedback/agent` Request body (JSON): ```json { "page_path": "/troubleshooting/aws-s3-secrets/", "page_title": "AWS S3 secrets troubleshooting", "text": "", "source": "" } ``` Only `page_path` and `text` are required. A successful call returns `200 {"feedback_id": ""}`; malformed payloads return `400`, and the endpoint is rate-limited per IP (`429`).