-
Notifications
You must be signed in to change notification settings - Fork 301
Open
Labels
documentationImprovements or additions to documentationImprovements or additions to documentation
Description
Documentation: Add Changelog Entry for v0.4.0 Release
Overview
The AWS SDK v2 upgrade (PR #683) represents a significant improvement to Litestream's S3 functionality. This needs to be properly documented in the changelog for the upcoming v0.4.0 release.
Problem
Users need to be informed about:
- The AWS SDK upgrade and its benefits
- New configuration options available
- Performance improvements
- Any migration considerations
Proposed Changelog Entry
## [0.4.0] - 2025-XX-XX
### Major Improvements
#### AWS SDK v2 Upgrade
Litestream now uses AWS SDK for Go v2 for all S3 operations, bringing significant improvements in performance, reliability, and functionality.
**Benefits:**
- **Better Performance**: Connection pooling and optimized HTTP client reduce latency
- **Improved Reliability**: Adaptive retry logic with exponential backoff (up to 10 attempts)
- **Enhanced Security**: Automatic CRC32 checksum validation for data integrity
- **Modern Authentication**: Better support for IAM roles, assume role, and credential chains
- **Reduced Memory Usage**: More efficient memory management for large uploads
**New Configuration Options:**
- `http-timeout`: Configure total timeout for S3 operations (default: 24h)
- `part-size`: Set multipart upload part size (default: 5MB, min: 5MB)
- `concurrency`: Control parallel part uploads (default: 5)
**Example Configuration:**
```yaml
replicas:
- type: s3
bucket: my-bucket
path: my-database
region: us-east-1
# New performance options
part-size: 20MB # Larger parts for big databases
concurrency: 10 # More parallel uploads
http-timeout: 6h # Custom timeout
Improvements
- S3 Error Handling: More descriptive error messages with operation context
- S3-Compatible Services: Better support for MinIO, Wasabi, and other S3-compatible storage
- Configuration Validation: Clearer error messages for misconfigured S3 settings
- Performance: Multipart uploads now configurable for optimal throughput
- Testing: Comprehensive test coverage for S3 operations including error scenarios
Bug Fixes
- Fixed S3 region detection when using IAM roles
- Improved handling of S3 bucket validation errors
- Better error reporting for authentication failures
Internal Changes
- Migrated from
github.com/aws/aws-sdk-go
github.com/aws/aws-sdk-go-v2
- Updated S3 client to use AWS SDK v2 configuration patterns
- Improved test infrastructure for S3 integration testing
- Added adaptive testing for moto mock server vs real S3
Compatibility
- No Breaking Changes: All existing S3 configurations continue to work
- Backward Compatible: No changes required to existing Litestream configurations
- Migration: Automatic - no user action required
Documentation
- Added AWS SDK v2 migration guide
- Added S3-compatible services configuration guide
- Added S3 performance tuning documentation
- Updated example configurations with new options
Known Issues
- Multipart upload testing with moto requires workaround for checksum validation (tracked in Update S3 multipart upload test once moto issue #8762 is resolved #700)
Contributors
Thanks to @corylanou for the AWS SDK v2 upgrade implementation and comprehensive testing improvements.
Upgrading
No configuration changes required. To use new performance options:
-
Update to v0.4.0:
# Docker docker pull litestream/litestream:0.4.0 # Binary wget https://github.com/benbjohnson/litestream/releases/download/v0.4.0/litestream-v0.4.0-linux-amd64.tar.gz
-
(Optional) Add performance tuning:
replicas: - type: s3 part-size: 20MB concurrency: 10
-
Restart Litestream service
References
- PR feat: upgrade AWS SDK from v1 to v2 for S3 client #683: AWS SDK v2 Upgrade
- Issue Update S3 multipart upload test once moto issue #8762 is resolved #700: Moto multipart upload test tracking
- AWS SDK v2 Documentation: https://aws.github.io/aws-sdk-go-v2/
### Additional Release Notes Sections to Update
#### README.md Updates
```markdown
### What's New in v0.4.0
- **AWS SDK v2**: Improved S3 performance and reliability
- **Configurable Multipart Uploads**: Tune S3 uploads for your needs
- **Better S3-Compatible Support**: Enhanced compatibility with MinIO, Wasabi, etc.
Documentation Site Updates
- Add banner announcing v0.4.0 with SDK v2 improvements
- Update S3 configuration page with new options
- Add performance tuning guide to navigation
Implementation Steps
- Add changelog entry to CHANGELOG.md
- Update README.md with version highlights
- Create GitHub release with detailed notes
- Update documentation site with new version
- Announce in Litestream Slack channel
Priority
High - Required for v0.4.0 release
Labels
- documentation
- changelog
- release
- v0.4.0
Metadata
Metadata
Assignees
Labels
documentationImprovements or additions to documentationImprovements or additions to documentation