Nslookup does not work in latest busybox image

[krishshenoy]

I deployed a kubernetes image using the latest version of busybox image.
After the pod was successfully deployed I tried to run
kubectl exec busybox nslookup kubernetes.default

The nslookup command no longer works.

shenoyk-m01:image-pipeline shenoyk$ kubectl exec busybox nslookup kubernetes.default
Server: 10.0.0.10
Address: 10.0.0.10:53

** server can't find kubernetes.default: NXDOMAIN

*** Can't find kubernetes.default: No answer

The same command works when specifying busybox:1.28 version for the image. Nslookup started failing with the latest version

busybox.yaml is below.

apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:

• image: busybox
  command:
  • sleep
  • "3600"
    imagePullPolicy: Always
    name: busybox
    restartPolicy: Always

[wglambert]

Seems to be a kubernetes configuration issue

Not able to reproduce the issue through Docker standalone

$ docker run --rm -dit --name busybox busybox:latest
$ docker exec -it busybox sh

# ping google.com
PING google.com (172.217.11.174): 56 data bytes
64 bytes from 172.217.11.174: seq=0 ttl=53 time=14.993 ms
64 bytes from 172.217.11.174: seq=1 ttl=53 time=14.598 ms
64 bytes from 172.217.11.174: seq=2 ttl=53 time=14.039 ms
^C
# nslookup github.com
Server:    8.8.8.8
Address 1: 8.8.8.8 google-public-dns-a.google.com

Name:      github.com
Address 1: 192.30.255.112 lb-192-30-255-112-sea.github.com
Address 2: 192.30.255.113 lb-192-30-255-113-sea.github.com
# nslookup google.com
Server:    8.8.8.8
Address 1: 8.8.8.8 google-public-dns-a.google.com

Name:      google.com
Address 1: 2607:f8b0:4007:804::200e lax28s15-in-x0e.1e100.net
Address 2: 216.58.219.14 lax17s03-in-f14.1e100.net

Kubernetes with hostNetwork: true

$ kubectl exec busybox-7cc555b5d6-2mmcr ping google.com
PING google.com (172.217.11.174): 56 data bytes
64 bytes from 172.217.11.174: seq=0 ttl=54 time=13.444 ms
64 bytes from 172.217.11.174: seq=1 ttl=54 time=14.249 ms
64 bytes from 172.217.11.174: seq=2 ttl=54 time=20.149 ms
^C

$ kubectl exec busybox-7cc555b5d6-2mmcr nslookup google.com 8.8.8.8
Server:         8.8.8.8
Address:        8.8.8.8:53

Non-authoritative answer:
Name:   google.com
Address: 172.217.11.174

*** Can't find google.com: No answer

$ kubectl exec busybox-7cc555b5d6-2mmcr nslookup kubernetes.default 8.8.8.8
Server:         8.8.8.8
Address:        8.8.8.8:53

** server can't find kubernetes.default: NXDOMAIN

*** Can't find kubernetes.default: No answer

$ kubectl exec busybox-7cc555b5d6-2mmcr nslookup kubernetes.default
Server:         127.0.0.53
Address:        127.0.0.53:53

** server can't find kubernetes.default: NXDOMAIN

*** Can't find kubernetes.default: No answer

This seems to be the most relevant issue I found kubernetes/kubernetes#33798

[tianon]

This reminds me of the fun we had back in #9, but that doesn't seem related. 😞

[krishshenoy]

I have a kubernetes cluster monitoring test that continually deploys a busybox pod in a cluster and verifies DNS resolution within the pod by executing kubectl exec nslookup. It started failing right when I downloaded the latest busybox image. Installing a busybox pod with the previous version 1.28 of the image nslookup works. All signs point to a change in this latest version that is causing the failure.

[tianon]

Unfortunately, that only narrows it down to somewhere in the sea of 438 files changed, 9453 insertions(+), 4480 deletions(-) (from 1_28_4 to 1_29_1 in the Git tags of the two releases).

[tianon]

Something in here seems most likely:

$ git log --oneline 1_28_4...1_29_1 -- networking/nslookup.c
2f7738e47 nslookup: placate "warning: unused variable i"
c72499584 nslookup: simplify make_ptr
71e4b3f48 nslookup: get rid of query::rlen field
58e43a4c4 nslookup: move array of queries to "globals"
4b6091f92 nslookup: accept lowercase -type=soa, document query types
6cdc3195a nslookup: change -stats to -debug (it's a bug in bind that it accepts -s)
d4461ef9f nslookup: rework option parsing
a980109c6 nslookup: smaller qtypes[] array
2cf75b3c8 nslookup: process replies immediately, do not store them
4e73c0f65 nslookup: fix output corruption for "nslookup 1.2.3.4"
cf950cd3e nslookup: more closely resemble output format of bind-utils-9.11.3
71e016d80 nslookup: shrink send_queries()
db93b21ec nslookup: use xmalloc_sockaddr2dotted() instead of homegrown function
55bc8e882 nslookup: usee bbox network functions instead of opne-coded mess
0dd3be8c0 nslookup: add openwrt / lede version

[djsly]

/label bug
we are having the same issue.
1.27/1.28 are working , 1.29/1.29.1 are not

kubectl run --attach busybox --rm --image=busybox:1.27 --restart=Never -- sh -c "sleep 4 && nslookup kubernetes.default"
If you don't see a command prompt, try pressing enter.

Server:    192.168.0.10
Address 1: 192.168.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default
Address 1: 192.168.0.1 kubernetes.default.svc.cluster.local

kubectl run --attach busybox --rm --image=busybox:1.28 --restart=Never -- sh -c "sleep 4 && nslookup kubernetes.default"
If you don't see a command prompt, try pressing enter.

Server:    192.168.0.10
Address 1: 192.168.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default
Address 1: 192.168.0.1 kubernetes.default.svc.cluster.local

 kubectl run --attach busybox --rm --image=busybox:1.29 --restart=Never -- sh -c "sleep 4 && nslookup kubernetes.default"
If you don't see a command prompt, try pressing enter.

Server:         192.168.0.10
Address:        192.168.0.10:53

** server can't find kubernetes.default: NXDOMAIN

*** Can't find kubernetes.default: No answer

 kubectl run --attach busybox --rm --image=busybox:1.29.1 --restart=Never -- sh -c "sleep 4 && nslookup kubernetes.default"
If you don't see a command prompt, try pressing enter.


Server:         192.168.0.10
Address:        192.168.0.10:53

** server can't find kubernetes.default: NXDOMAIN

*** Can't find kubernetes.default: No answer

[tianon]

Anyone filed an issue with upstream yet? Or done more digging to figure out exactly which upstream change caused it? The only change was the BusyBox version bump, no uclibc update, buildroot update, etc, so this is firmly an upstream bug and one specially BusyBox upstream.

[djsly]

https://bugs.busybox.net/show_bug.cgi?id=11161

[tokiwinter]

Same issue here. Reverting to 1.28 fixed the issue for me.

[tianon]

How does this relate to #27? Are they the same issue?

[tianon]

From what I can tell, the new resolver in BusyBox's nslookup doesn't support DNS search domains at all, which seems like a pretty hefty regression.

[krishshenoy]

Thanks tianon. How will this be addressed?