Skip to content

IPv6-only support for S3 clients #4580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 24, 2025
Merged

Conversation

TheanLim
Copy link
Contributor

@TheanLim TheanLim commented Apr 18, 2025

Summary

Enables S3 and S3Manager clients to automatically use dualstack endpoints when running on IPv6-only instances.

Implementation details

  1. Client Configuration Updates
    1. Modified NewS3ManagerClient and NewS3Client to accept IPCompatibility parameter
    2. Enhanced createAwsConfig to set useDualStackEndpoint based on IPv6 compatibility status
  2. Task Resources
    1. S3 clients are mainly used by TaskResources such as EnvFiles, Firelens, and CredentialSpecs.
    2. Added an ipCompatibility field to each task resource.
      1. Updated New* constructors to accept an additional ipCompatibility parameter. New* is invoked when there's a new incoming task that requires task resources.
      2. Config.InstanceIPCompatibility is the source of value
  3. Modified Task Resources interface to pass in *Config to Initialize(). Initialize() is invoked when Agent restarts and reinitialize task resouces
    1. Update all other task resources that implement this interface
  4. CredentialsMetadataSetter Updates
    1. Updated NewCredentialsMetadataSetter to accept an IPCompatibility parameter
    2. Modified checkAndSetDomainlessGMSATaskExecutionRoleCredentialsImpl to use the IPCompatibility parameter when creating the CredentialSpecResource

Testing

Executed functional test suite on dualstack subnet. Verified all tests passed and dualstack endpoint usage through docker container logs.

Host: {bucket-name}-prod.s3.dualstack.us-west-2.amazonaws.com

Setups done:

  • Set IPCompatibility to NewIPv6OnlyCompatibility
  • Pin the functional test suite on a dualstack subnet (agent is currently connecting to control plane via IPv4).
  • Set the awsConfig to Debug level and directed logs to os.Stdout.

New tests cover the changes: yes

  • Refactored TestCreateAWSConfig to remove redundant test
  • Extended the test to verify correct Dualstack flag configuration in Config

Description for the changelog

Enhancement: S3 and S3Manager clients resolves to dualstack endpoint on IPV6-only instances

Additional Information

Does this PR include breaking model changes? If so, Have you added transformation functions?

Does this PR include the addition of new environment variables in the README?

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@TheanLim TheanLim requested a review from a team as a code owner April 18, 2025 16:46
@TheanLim TheanLim force-pushed the thean/s3-ipv6-client-flag branch from d23de2f to c41b739 Compare April 21, 2025 15:58
@TheanLim TheanLim changed the title [WIP] Thean/s3 ipv6 client flag [WIP] IPv6-only support for s3 clients Apr 21, 2025
@TheanLim TheanLim force-pushed the thean/s3-ipv6-client-flag branch from c41b739 to 411054c Compare April 21, 2025 18:14
@TheanLim TheanLim changed the title [WIP] IPv6-only support for s3 clients IPv6-only support for S3 clients Apr 21, 2025
@TheanLim TheanLim enabled auto-merge April 21, 2025 19:06
@TheanLim TheanLim force-pushed the thean/s3-ipv6-client-flag branch 2 times, most recently from 9477d45 to 5d08a22 Compare April 22, 2025 22:15
@TheanLim TheanLim disabled auto-merge April 22, 2025 22:18
@TheanLim TheanLim force-pushed the thean/s3-ipv6-client-flag branch from 5d08a22 to 12e06af Compare April 22, 2025 22:23
amogh09
amogh09 previously approved these changes Apr 22, 2025
singholt
singholt previously approved these changes Apr 23, 2025
@TheanLim TheanLim force-pushed the thean/s3-ipv6-client-flag branch 5 times, most recently from cd1ac65 to ddd8de7 Compare April 24, 2025 00:35
@TheanLim TheanLim disabled auto-merge April 24, 2025 00:38
@TheanLim TheanLim force-pushed the thean/s3-ipv6-client-flag branch from ddd8de7 to e3eb5d2 Compare April 24, 2025 00:41
@TheanLim TheanLim force-pushed the thean/s3-ipv6-client-flag branch from e3eb5d2 to 85b212f Compare April 24, 2025 04:23
@TheanLim TheanLim force-pushed the thean/s3-ipv6-client-flag branch from 85b212f to 2b6d42a Compare April 24, 2025 04:34
@TheanLim TheanLim requested review from amogh09 and singholt April 24, 2025 05:46
@TheanLim TheanLim enabled auto-merge (squash) April 24, 2025 05:57
@TheanLim TheanLim merged commit 5037d0d into aws:dev Apr 24, 2025
40 checks passed
timj-hh pushed a commit to timj-hh/amazon-ecs-agent that referenced this pull request Jul 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants