Fault Injection - Multi-ENI detection for host mode tasks & IPv6-only task detection for all tasks #4670

amogh09 · 2025-06-03T21:48:02Z

Summary

This PR modifies the behavior of the GetTaskMetadataWithTaskNetworkConfig method of TMDSAgentState implementation to:

Return task ENI's IPv4 and IPv6 addresses for awsvpc mode tasks
Look up and return default IPv4 and IPv6 network interfaces and their IP addresses in the host network namespace for host mode tasks

The definition of IPv6-only tasks in the context of Fault Injection handlers has been updated accordingly. The new definition states that an IPv6-only task has a single network interface with no IPv4 addresses and at least one IPv6 address. This updated definition correctly identifies host mode tasks with separate default network interfaces for IPv4 and IPv6 as not IPv6-only. The isIPv6OnlyTask function has been updated to reflect this change.

Implementation details

TMDSAgentState.getTaskMetadata method that's responsible for populating task network configuration for all tasks on all platforms is updated so that it populates IP addresses for awsvpc tasks.
TMDSAgentState.GetTaskMetadataWithTaskNetworkConfig method for Linux specifically is updated so that for host mode tasks it looks up default network interfaces for IPv4 and IPv6 using the GetDefaultNetworkInterfaces utility function that was introduced in Net utils for finding default network interfaces #4665.
isIPv6OnlyTask function in ecs-agent/tmds/handlers/fault/v1/handlers/handlers.go is updated as per the new definition mentioned above.
Unit test updates.

Testing

Launched two instances, one IPv6-only and another one with two ENIs - IPv4-only and IPv6-only (we will call this v4v6 instance). On both instances I temporarily renamed ip6tables utility so that it is not found by Agent.

On v4v6 instance, verified that start blackhole port fault request do not fail but the fault is not injected for IPv6 traffic.

bash-5.2# curl -XPOST --data '{"Port": 80, "Protocol": "tcp", "TrafficType": "egress"}' ${ECS_AGENT_URI}/fault/v1/network-blackhole-port/start
{"Status":"running"}bash-5.2#
bash-5.2# curl -4 --connect-timeout 2 -w "IP: %{remote_ip}\nHTTP: %{response_code}\n" -s example.com -o /dev/null
IP:
HTTP: 000
bash-5.2# curl -6 --connect-timeout 2 -w "IP: %{remote_ip}\nHTTP: %{response_code}\n" -s example.com -o /dev/null
IP: 2600:1406:bc00:53::b81e:94c8
HTTP: 200

On the IPv6-only instance this causes an internal server error.

bash-5.2# curl -XPOST --data '{"Port": 80, "Protocol": "tcp", "TrafficType": "egress"}' ${ECS_AGENT_URI}/fault/v1/network-blackhole-port/start
{"Error":"internal error"}bash-5.2#
bash-5.2# curl -XPOST --data '{"Port": 80, "Protocol": "tcp", "TrafficType": "egress"}' ${ECS_AGENT_URI}/fault/v1/network-blackhole-port/stop
{"Error":"internal error"}bash-5.2# exit

Logs on IPv6-only instance -

[ec2-user@ipv6only ~]$ grep -i ip6tables /var/log/ecs/ecs-agent.log
level=error time=2025-06-03T23:07:28Z msg="Unable to execute the command" command="ip6tables -w 5 -N egress-tcp-80" output="" taskArn="arn:aws:ecs:us-west-2:979604884904:task/sc-test/9fa2dec442f34773b1e467eb7e607fba" error="exec: \"ip6tables\": executable file not found in $PATH" netns="host"
level=error time=2025-06-03T23:07:53Z msg="Unable to execute the command" netns="host" command="ip6tables -w 5 -F egress-tcp-80" output="" taskArn="arn:aws:ecs:us-west-2:979604884904:task/sc-test/9fa2dec442f34773b1e467eb7e607fba" error="exec: \"ip6tables\": executable file not found in $PATH"

New tests cover the changes: yes

Description for the changelog

enhancement: [TMDS Fault Injection] Improve network interface detection to handle multiple default interfaces in host mode and update IPv6-only task identification

Additional Information

Does this PR include breaking model changes? If so, Have you added transformation functions?

No

Does this PR include the addition of new environment variables in the README?

No

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…ple default interfaces in host mode and update IPv6-only task identification

ecs-agent/tmds/handlers/fault/v1/handlers/handlers.go

mye956 · 2025-06-04T16:14:48Z

agent/handlers/v4/tmdsstate.go

@@ -21,6 +21,7 @@ import (
 	"github.com/aws/amazon-ecs-agent/ecs-agent/logger"
 	"github.com/aws/amazon-ecs-agent/ecs-agent/logger/field"
 	tmdsv4 "github.com/aws/amazon-ecs-agent/ecs-agent/tmds/handlers/v4/state"
+	"github.com/pkg/errors"


nit: could better format the imports here

Right.. let me do this as a follow-up.

https://github.com/daixiang0/gci can help you regroup imports which can be integrated into the repo if needed.

Addressed in #4671

mye956 · 2025-06-04T16:15:22Z

agent/handlers/v4/tmdsstate.go

 		} else {
-			taskNetworkConfig = tmdsv4.NewTaskNetworkConfig(task.GetNetworkMode(), task.GetNetworkNamespace(), task.GetDefaultIfname())
+			// For bridge mode there is no concept of task network interfaces in ECS


also true for "none" network mode as well

Right.. let me do this as a follow-up.

Addressed in #4671

xxx0624 · 2025-06-04T18:16:40Z

agent/handlers/v4/tmdsstate.go

@@ -21,6 +21,7 @@ import (
 	"github.com/aws/amazon-ecs-agent/ecs-agent/logger"
 	"github.com/aws/amazon-ecs-agent/ecs-agent/logger/field"
 	tmdsv4 "github.com/aws/amazon-ecs-agent/ecs-agent/tmds/handlers/v4/state"
+	"github.com/pkg/errors"


https://github.com/daixiang0/gci can help you regroup imports which can be integrated into the repo if needed.

…ple default interfaces in host mode and update IPv6-only task identification (aws#4670)

Fault Injection - Improve network interface detection to handle multi…

a6fd887

…ple default interfaces in host mode and update IPv6-only task identification

amogh09 force-pushed the tmds-multi-eni branch from 5a461d0 to a6fd887 Compare June 3, 2025 22:18

amogh09 added the bot/test label Jun 3, 2025

amazon-ecs-bot removed the bot/test label Jun 3, 2025

amogh09 changed the title ~~Tmds multi eni~~ Fix network interface detection for host mode tasks in fault injection handlers Jun 3, 2025

amogh09 marked this pull request as ready for review June 3, 2025 23:04

amogh09 requested a review from a team as a code owner June 3, 2025 23:04

amogh09 changed the title ~~Fix network interface detection for host mode tasks in fault injection handlers~~ Support for multi-ENI host mode tasks in fault injection handlers Jun 3, 2025

amogh09 changed the title ~~Support for multi-ENI host mode tasks in fault injection handlers~~ Fault Injection - Multi-ENI detection for host mode tasks; IPv6-only task detection for all tasks Jun 3, 2025

amogh09 changed the title ~~Fault Injection - Multi-ENI detection for host mode tasks; IPv6-only task detection for all tasks~~ Fault Injection - Multi-ENI detection for host mode tasks & IPv6-only task detection for all tasks Jun 3, 2025

mye956 reviewed Jun 4, 2025

View reviewed changes

ecs-agent/tmds/handlers/fault/v1/handlers/handlers.go Show resolved Hide resolved

mye956 reviewed Jun 4, 2025

View reviewed changes

mye956 approved these changes Jun 4, 2025

View reviewed changes

xxx0624 approved these changes Jun 4, 2025

View reviewed changes

amogh09 merged commit c1c8f09 into aws:dev Jun 4, 2025
40 of 41 checks passed

mye956 mentioned this pull request Jun 9, 2025

Release 1.95.0 #4673

Merged

prateekchaudhry mentioned this pull request Jul 3, 2025

Release 1.96.0 #4699

Merged

timj-hh pushed a commit to timj-hh/amazon-ecs-agent that referenced this pull request Jul 19, 2025

Fault Injection - Improve network interface detection to handle multi…

e84a53a

…ple default interfaces in host mode and update IPv6-only task identification (aws#4670)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fault Injection - Multi-ENI detection for host mode tasks & IPv6-only task detection for all tasks #4670

Fault Injection - Multi-ENI detection for host mode tasks & IPv6-only task detection for all tasks #4670

Uh oh!

amogh09 commented Jun 3, 2025 •

edited

Loading

Uh oh!

Uh oh!

mye956 Jun 4, 2025

Uh oh!

amogh09 Jun 4, 2025

Uh oh!

xxx0624 Jun 4, 2025 •

edited

Loading

Uh oh!

amogh09 Jun 4, 2025

Uh oh!

mye956 Jun 4, 2025

Uh oh!

amogh09 Jun 4, 2025

Uh oh!

amogh09 Jun 4, 2025

Uh oh!

xxx0624 Jun 4, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Fault Injection - Multi-ENI detection for host mode tasks & IPv6-only task detection for all tasks #4670

Fault Injection - Multi-ENI detection for host mode tasks & IPv6-only task detection for all tasks #4670

Uh oh!

Conversation

amogh09 commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Implementation details

Testing

Description for the changelog

Additional Information

Licensing

Uh oh!

Uh oh!

mye956 Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

amogh09 Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

xxx0624 Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amogh09 Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

mye956 Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

amogh09 Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

amogh09 Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

xxx0624 Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

amogh09 commented Jun 3, 2025 •

edited

Loading

xxx0624 Jun 4, 2025 •

edited

Loading

xxx0624 Jun 4, 2025 •

edited

Loading