S3 Lifecycle Policies
Key concepts
Transition actions
Expiration actions
Filter by prefix/tags
Multipart upload cleanup
Versioning considerations
Overview
Amazon S3 Lifecycle policies enable automated management of objects throughout their lifecycle, transitioning data to lower-cost storage classes and automatically expiring objects based on defined rules. This is a critical topic for the SAA-C03 exam.
Core Concept
S3 Lifecycle configurations consist of rules that define transition actions (move to cheaper storage classes) and expiration actions (delete objects). Rules can be applied to all objects, by prefix, or by object tags. Lifecycle rules run once per day asynchronously.
Lifecycle policies can 'transition' or 'expire' objects, but NOT 'protect' them. For protection from deletion, use S3 Object Lock. This distinction appears frequently in exam questions.
Key Concepts
Lifecycle Action Types
Transition Actions
Purpose: Automatically move objects to lower-cost storage classes based on age
How It Works:
- Define number of days after object creation
- Objects transition to specified storage class
- Can chain multiple transitions (Standard → IA → Glacier)
Supported Transitions:
S3 Standard → Any lower tier
S3 Standard-IA → One Zone-IA, Glacier tiers
S3 Intelligent-Tiering → Glacier tiers only
S3 One Zone-IA → Glacier Flexible/Deep Archive only
Constraints:
- Minimum 30 days before transition to IA classes
- Cannot transition to S3 Standard or Express One Zone
- Objects < 128 KB not recommended for IA transitions
Expiration Actions
Purpose: Automatically delete objects after a specified period
How It Works:
- Define number of days or specific date for expiration
- S3 deletes objects on your behalf (asynchronously)
- Commonly used for log files, temporary data
Behavior by Bucket State: | Bucket Versioning | Expiration Behavior | |-------------------|---------------------| | Disabled | Object permanently deleted | | Enabled | Delete marker added (object becomes noncurrent) | | Suspended | Delete marker added with null version ID |
Use Cases:
- Regulatory compliance (delete after retention period)
- Log file cleanup
- Temporary data management
Multipart Upload Cleanup
Purpose: Abort and clean up incomplete multipart uploads
The Problem:
- Incomplete multipart uploads consume storage and incur charges
- Uploads not visible in normal bucket listings
- Can accumulate over time, wasting money
Configuration:
{
"AbortIncompleteMultipartUpload": {
"DaysAfterInitiation": 7
}
}
Key Limitation: Cannot use object tag filters with this action
Best Practice: Apply to all buckets with multipart upload activity (typically 7 days is a safe default)
Versioning Considerations
Noncurrent Version Management
Purpose: Manage previous versions of objects in versioned buckets
Available Actions:
-
NoncurrentVersionTransition
- Transition noncurrent versions to cheaper storage
- Configure days since object became noncurrent
- Optionally specify number of versions to retain (1-100)
-
NoncurrentVersionExpiration
- Permanently delete noncurrent versions
- Based on days since becoming noncurrent
- Can specify maximum versions to retain
Example Configuration:
{
"NoncurrentVersionTransition": {
"NoncurrentDays": 30,
"StorageClass": "GLACIER",
"NewerNoncurrentVersions": 3
},
"NoncurrentVersionExpiration": {
"NoncurrentDays": 365,
"NewerNoncurrentVersions": 1
}
}
Expired Object Delete Markers
Purpose: Clean up delete markers that no longer have associated versions
What Is It?
- A delete marker with zero noncurrent versions
- Created when all previous versions are deleted/expired
- Does NOT incur storage charges
- CAN impact LIST operation performance
Cleanup Action:
{
"Expiration": {
"ExpiredObjectDeleteMarker": true
}
}
When to Use: Buckets with high churn where objects are frequently deleted and expire
Lifecycle Rule Configuration
{
"Rules": [
{
"ID": "LogFileManagement",
"Status": "Enabled",
"Filter": {
"Prefix": "logs/"
},
"Transitions": [
{
"Days": 30,
"StorageClass": "STANDARD_IA"
},
{
"Days": 90,
"StorageClass": "GLACIER"
}
],
"Expiration": {
"Days": 365
}
},
{
"ID": "CleanupMultipartUploads",
"Status": "Enabled",
"Filter": {
"Prefix": ""
},
"AbortIncompleteMultipartUpload": {
"DaysAfterInitiation": 7
}
},
{
"ID": "VersionManagement",
"Status": "Enabled",
"Filter": {
"Prefix": "documents/"
},
"NoncurrentVersionTransitions": [
{
"NoncurrentDays": 30,
"StorageClass": "STANDARD_IA"
}
],
"NoncurrentVersionExpiration": {
"NoncurrentDays": 90,
"NewerNoncurrentVersions": 5
}
}
]
}# View current lifecycle configuration
aws s3api get-bucket-lifecycle-configuration \
--bucket my-bucket
# Apply lifecycle configuration from file
aws s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration file://lifecycle.json
# Delete lifecycle configuration
aws s3api delete-bucket-lifecycle \
--bucket my-bucket
# List incomplete multipart uploads (to verify cleanup)
aws s3api list-multipart-uploads \
--bucket my-bucketTransition Constraints
Minimum Days Between Transitions
| From Storage Class | To Storage Class | Minimum Days |
|---|---|---|
| S3 Standard | S3 Standard-IA | 30 |
| S3 Standard | S3 One Zone-IA | 30 |
| S3 Standard | S3 Intelligent-Tiering | 0 |
| S3 Standard | S3 Glacier Instant | 0 |
| S3 Standard | S3 Glacier Flexible | 0 |
| S3 Standard | S3 Glacier Deep Archive | 0 |
| S3 Standard-IA | S3 One Zone-IA | 30 |
| S3 Standard-IA | S3 Glacier (any) | 0 |
| S3 Intelligent-Tiering | S3 Glacier Flexible | 0 |
| S3 Intelligent-Tiering | S3 Glacier Deep Archive | 0 |
Best Practices
- Analyze Access Patterns First: Use S3 Storage Class Analysis for at least 30 days before defining transition rules
- Start Simple, Then Scale: Begin with basic rules and add complexity as you understand your data patterns
- Test in Non-Production: Validate lifecycle rules on test data before applying to production buckets
- Document Your Policies: Maintain documentation explaining the business rationale for each rule
- Clean Up Incomplete Uploads: Always include an AbortIncompleteMultipartUpload rule (7 days recommended)
- Manage Versions Proactively: Use NoncurrentVersionExpiration to prevent version accumulation
- Review Regularly: Access patterns change; review and update policies quarterly
Common Exam Scenarios
Exam Scenario Decision Guide
| Scenario | Recommended Configuration | Key Reasoning |
|---|---|---|
| Log files accessed frequently for 30 days, rarely after | Transition to Standard-IA at 30 days, Glacier at 90 days, expire at 365 days | Matches access pattern with cost optimization |
| Compliance data must be retained 7 years, never accessed | Transition to Glacier Deep Archive immediately, expire at 2555 days | Lowest cost for long-term retention |
| Backup files with unpredictable access | Transition to Intelligent-Tiering immediately | Automatic tier management without retrieval fees |
| Temporary uploads that may fail | AbortIncompleteMultipartUpload after 7 days | Prevents orphaned upload parts from accumulating |
| Versioned documents with 5 versions needed | NoncurrentVersionExpiration with NewerNoncurrentVersions: 5 | Retains required versions, cleans up older ones |
| Application generates many small files | Keep in S3 Standard or Intelligent-Tiering | Small object overhead makes IA classes inefficient |
| Compliance requires 30-day recovery window then archive | Transition current to Glacier at 30 days, noncurrent versions expire at 30 days | Balances recovery needs with cost |
| High-churn bucket with many deletions | ExpiredObjectDeleteMarker: true | Improves LIST performance by cleaning markers |
Common Pitfalls
Minimum Storage Duration Charges
Transitioning or deleting objects before their minimum storage duration (30 days for IA, 90 days for Glacier Instant/Flexible, 180 days for Deep Archive) incurs pro-rated charges. Lifecycle rules don't prevent these charges—they still apply.
Cannot Override with Bucket Policy
Bucket policies CANNOT prevent lifecycle rule deletions or transitions. Even with an explicit deny for all principals, lifecycle rules execute normally. Use S3 Object Lock for immutable data protection.
Small Object Inefficiency
Objects smaller than 128 KB are charged as 128 KB in IA and Glacier Instant classes. Additionally, minimum storage duration applies. For many small files, the lifecycle transition may cost more than keeping objects in Standard.
Lifecycle Rules Run Daily
S3 runs lifecycle rules once per day (not continuously). There can be up to a 24-48 hour delay between when an object becomes eligible and when the action occurs. Don't rely on immediate transitions.
Multipart Cleanup Tag Limitation
The AbortIncompleteMultipartUpload action CANNOT use object tag filters—only prefix filters or apply to entire bucket. Plan your bucket structure accordingly if you need selective multipart cleanup.
Related Services
Quick Reference
Lifecycle Configuration Limits
| Limit | Value |
|---|---|
| Maximum rules per bucket | 1,000 |
| Maximum prefix length | 1,024 characters |
| Minimum transition to IA classes | 30 days |
| Rule evaluation frequency | Once per day |
| Maximum versions to retain | 100 |
Key CLI Commands
# Get lifecycle configuration
aws s3api get-bucket-lifecycle-configuration --bucket BUCKET_NAME
# Put lifecycle configuration
aws s3api put-bucket-lifecycle-configuration \
--bucket BUCKET_NAME \
--lifecycle-configuration file://lifecycle.json
# Delete lifecycle configuration
aws s3api delete-bucket-lifecycle --bucket BUCKET_NAME
# List incomplete multipart uploads
aws s3api list-multipart-uploads --bucket BUCKET_NAME
# Abort specific multipart upload
aws s3api abort-multipart-upload \
--bucket BUCKET_NAME \
--key OBJECT_KEY \
--upload-id UPLOAD_ID
Filter Types
| Filter Type | Use Case | Example |
|---|---|---|
| Prefix | Path-based rules | "Prefix": "logs/" |
| Tag | Content-based rules | "Tag": {"Key": "archive", "Value": "true"} |
| Combined (AND) | Specific subsets | Prefix AND Tag combined |
| Object Size | Size-based rules | "ObjectSizeGreaterThan": 131072 |
Test Your Knowledge
A company stores application logs in S3. Logs are accessed frequently for 7 days, occasionally for 30 days, and must be retained for 1 year for compliance. Which lifecycle configuration minimizes costs while meeting requirements?
A versioned S3 bucket accumulates many noncurrent object versions, increasing costs. The company wants to keep only the 3 most recent noncurrent versions and delete older ones after 90 days. Which lifecycle action achieves this?
An S3 bucket receives many multipart uploads from an application. Some uploads fail and leave incomplete parts. The operations team notices storage costs are higher than expected. What is the MOST cost-effective solution?
A company has a lifecycle rule that transitions objects to S3 Glacier Flexible Retrieval after 30 days. They delete an object on day 25. What charges apply?