Zabbix DNS caching

DNS resolution started to feel a bit laggy recently, most notable on iOS devices. When I went down the troubleshooting rabbit hole, I found a large number of DNS queries from the Zabbix monitoring infrastructure.

Of the 37 hosts that are monitored by Zabbix, only 16 have DNS lookups for the hostnames.
This caused multiple lookups per second for each host - thousands of entries per 10 minutes.

A relatively easy fix was to install dnsmasq on the Zabbix server and enabling minimum caching of 10 minutes.

This post was a good start -

Zabbix is Secretly a DNS Stress Tester - blog.frankwayne.com

Zabbix is a great, open-source monitoring server, but it has no DNS cache. Here is how to calm the storm of DNS requests.

blog.frankwayne.comfwaynefrankwayne.com

The basic config helped. Looking at the title bar you'll see the drop in DNS lookups, but still getting about 170 / minute.

I added a minimum cache ttl, as the hosts on this network don't change IP often. Worst case, monitoring will break for 10 mins before dnsmasq resolves the IP again. Now there's a significant reduction in lookups - mobile performance is yet to be checked!

Ultimately my config file looked like this:

server=10.130.15.100

domain-needed
bogus-priv

# Bind only the local loopback address
listen-address=127.0.0.1
bind-interfaces
# Prevent dnsmasq from reading any files (like resolv.conf) to get DNS servers
no-resolv
no-poll
# Other settings automatically set on my distro
port=53
conf-dir=/etc/dnsmasq.d,.rpmnew,.rpmsave,.rpmorig
#XGA; add TTL as responses were TTL=0
min-cache-ttl=600