Latency and packet loss faults should apply to all network interfaces

[amogh09]

Summary

This PR changes network latency and packet loss Fault Injection handlers so that they apply to all network interfaces of the task instead of just the first one. This is relevant for host mode task on hosts with different default network interfaces for IPv4 and IPv6.

Implementation details

Fault injection handlers now loop through all network interfaces instead of looking at just the first one.

Testing

Launched an instance with two ENIs - one IPv4-only and one IPv6-only. So, this instance has different IPv4 and IPv6 default network interfaces.

Latency fault worked on both IPv4 and IPv6 targets as expected -

bash-5.2# curl -XPOST --data '{"DelayMilliseconds": 500, "JitterMilliseconds": 0, "Sources": ["23.215.0.138/24", "2600:1406:bc00:53::b81e:94ce/64"], "SourcesToFilter": ["2600:1406:bc00:53::b81e:94c8"]}' ${ECS_AGENT_URI}/fault/v1/network-latency/start
{"Status":"running"}bash-5.2#
bash-5.2# ping -c3 23.215.0.138
PING 23.215.0.138 (23.215.0.138) 56(84) bytes of data.
64 bytes from 23.215.0.138: icmp_seq=1 ttl=48 time=564 ms
64 bytes from 23.215.0.138: icmp_seq=2 ttl=48 time=564 ms
64 bytes from 23.215.0.138: icmp_seq=3 ttl=48 time=564 ms

--- 23.215.0.138 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 563.583/563.612/563.662/0.035 ms
bash-5.2# ping -c3 23.215.0.136
PING 23.215.0.136 (23.215.0.136) 56(84) bytes of data.
64 bytes from 23.215.0.136: icmp_seq=1 ttl=48 time=566 ms
64 bytes from 23.215.0.136: icmp_seq=2 ttl=48 time=566 ms
64 bytes from 23.215.0.136: icmp_seq=3 ttl=48 time=566 ms

--- 23.215.0.136 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 565.607/565.616/565.627/0.008 ms
bash-5.2# ping -c3 2600:1406:bc00:53::b81e:94ce
PING 2600:1406:bc00:53::b81e:94ce(2600:1406:bc00:53::b81e:94ce) 56 data bytes
64 bytes from 2600:1406:bc00:53::b81e:94ce: icmp_seq=1 ttl=50 time=533 ms
64 bytes from 2600:1406:bc00:53::b81e:94ce: icmp_seq=2 ttl=50 time=533 ms
64 bytes from 2600:1406:bc00:53::b81e:94ce: icmp_seq=3 ttl=50 time=533 ms

--- 2600:1406:bc00:53::b81e:94ce ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 533.428/533.433/533.436/0.003 ms
bash-5.2# ping -c3 2600:1406:bc00:53::b81e:94c8
PING 2600:1406:bc00:53::b81e:94c8(2600:1406:bc00:53::b81e:94c8) 56 data bytes
64 bytes from 2600:1406:bc00:53::b81e:94c8: icmp_seq=1 ttl=50 time=31.6 ms
64 bytes from 2600:1406:bc00:53::b81e:94c8: icmp_seq=2 ttl=50 time=31.6 ms
64 bytes from 2600:1406:bc00:53::b81e:94c8: icmp_seq=3 ttl=50 time=31.6 ms

--- 2600:1406:bc00:53::b81e:94c8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 31.610/31.622/31.647/0.017 ms

State of tc configuration with latency fault applied -

[ec2-user@ip-10-0-1-50 ~]$ tc q show dev ens5
qdisc prio 1: root refcnt 3 bands 3 priomap 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
qdisc netem 10: parent 1:1 limit 1000 delay 500ms
[ec2-user@ip-10-0-1-50 ~]$ tc q show dev ens6
qdisc prio 1: root refcnt 3 bands 3 priomap 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
qdisc netem 10: parent 1:1 limit 1000 delay 500ms

Packet loss fault worked on both IPv4 and IPv6 targets as expected -

bash-5.2# curl -XPOST --data '{"LossPercent": 33, "Sources": ["23.215.0.138/24", "2600:1406:bc00:53::b81e:94ce/64"], "SourcesToFilter": ["23.215.0.136"]}' ${ECS_AGENT_URI}/fault/v1/network-packet-loss/start
{"Status":"running"}bash-5.2#
bash-5.2# ping -i0.01 -c1000 23.215.0.138 | grep -A3 "ping statistics"
--- 23.215.0.138 ping statistics ---
1000 packets transmitted, 674 received, 32.6% packet loss, time 15925ms
rtt min/avg/max/mdev = 64.908/65.037/84.045/0.770 ms, pipe 5
bash-5.2# ping -i0.01 -c1000 23.215.0.136 | grep -A3 "ping statistics"
--- 23.215.0.136 ping statistics ---
1000 packets transmitted, 1000 received, 0% packet loss, time 15416ms
rtt min/avg/max/mdev = 63.741/63.903/90.693/1.220 ms, pipe 6
bash-5.2# ping -i0.01 -c1000 2600:1406:bc00:53::b81e:94ce | grep -A3 "ping statistics"
--- 2600:1406:bc00:53::b81e:94ce ping statistics ---
1000 packets transmitted, 669 received, 33.1% packet loss, time 15184ms
rtt min/avg/max/mdev = 33.346/33.402/35.425/0.114 ms, pipe 3

State of tc configuration with packet loss fault applied -

[ec2-user@ip-10-0-1-50 ~]$ tc q show dev ens5
qdisc prio 1: root refcnt 3 bands 3 priomap 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
qdisc netem 10: parent 1:1 limit 1000 loss 33%
[ec2-user@ip-10-0-1-50 ~]$ tc q show dev ens6
qdisc prio 1: root refcnt 3 bands 3 priomap 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
qdisc netem 10: parent 1:1 limit 1000 loss 33%

New tests cover the changes: yes

Description for the changelog

bugfix: Network latency and packet loss faults should apply to all default network interfaces in host mode

Additional Information

Does this PR include breaking model changes? If so, Have you added transformation functions?

No

Does this PR include the addition of new environment variables in the README?

No

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.