Amazon S3 + MemorySync

Overview

MemorySync's native S3 provider reads only the bucket paths an administrator explicitly approves, using read-only IAM credentials for the customer’s own bucket. Every exclusion — blocked key patterns, unsupported file types, archived storage classes, oversized objects, and objects unchanged since the last sync — is decided from the listing entry before any object is fetched, so a file that will not be indexed never costs a download. Because S3 emits no change notifications, a scheduled reconciliation pass detects objects deleted at the source and purges the memories derived from them.

Setup status and requirements

Supported version

Repository implementation reviewed 2026-08-07

Last setup review

2026-08-07

Permissions

A read-only IAM user with s3:ListBucket and s3:GetBucketLocation on the bucket, and s3:GetObject on its contents. Buckets encrypted with SSE-KMS also require kms:Decrypt: without it, listing succeeds and every download fails with a plain access denial. No paid plan or vendor review is required — the credentials belong to the customer’s own AWS account. MemorySync never writes to or deletes from the bucket.

Limits

Only paths an administrator approves are listed, and objects outside them are never read. Within an approved path, only file types the extractor registry can read are downloaded, up to a configured size limit; an allowlist is used rather than a denylist because the cost of guessing wrong is a paid download that yields nothing. Objects in Glacier or Deep Archive are skipped, because reading them requires an explicit restore. Keys resembling credentials — .env files, private keys, Terraform state, .ssh contents — are excluded for every tenant and cannot be re-enabled. S3 sends no change or deletion notifications, so new and removed objects are picked up on a scheduled sync and a periodic reconciliation rather than immediately. Object keys longer than the identifier column are stored under a deterministic digest, which keeps them syncable but not individually addressable by key. Memories are readable across the organization: object storage carries no per-user permission model to derive narrower access from.

Capabilities

• Per-path administrator approval
• Read-only IAM credentials verified before the connection is saved
• S3-compatible endpoints for R2, MinIO, Wasabi, and Backblaze B2
• Text extraction from PDF, Office, Markdown, and plain-text files
• Exclusions decided before download, so gated objects cost no egress
• ETag comparison so unchanged objects are never re-read
• Per-path incremental watermarks
• Scheduled reconciliation for deleted objects
• Credential redaction before indexing
• Key exclusion policy covering .env files, private keys, and Terraform state
• Skip reasons surfaced per path so a low count is explainable

Use Cases

Explore More