[dnstap] dnstap fanout and replay

[Tony Finch]

I have a short wishlist of dnstap-related tools. I haven't managed to find
out if anything like this already exists - if it does exist I'll be
grateful for any pointers!

We have a couple of kinds of people who have expressed interest in getting
dnstap feeds from our campus resolvers.

* There are people on site doing threat intelligence research, who would
  like a full feed of client queries and responses.

* And there are third parties who would like a passive DNS feed of
  outgoing resolver queries, and who aren't allowed a full-fat feed for
  privacy reasons.

The dnstap implementation in BIND only supports one output stream, so if
we are going to satisfy these consumers, we would need to split the dnstap
feed downstream of BIND before feeding the distributaries onwards.

More recently it occurred to me that it might be useful to generate
queries from a dnstap feed. I have a couple of scenarios:

* Replay client queries against a test server, to verify that it behaves
  OK with real-ish traffic. I have a tool for replaying cache dump files,
  but these are nothing like real user traffic since they don't include
  repeated queries etc.

* Replay resolver queries from a live server against a standby server.
  These queries are effectively the cache misses, so they are less costly
  to replicate than all the client traffic. This keeps the standby cache
  hot whereas at the moment my standby servers have cold caches.

  It might also be worth duplicating this traffic from one live server to
  the other one, in the hope that this increases the cache hit rate, since
  hit rate increses the more users a cache has. (Some experimentation
  needed!)

I'm not really insterested in the responses to these queries so it's OK if
the replay just drops the answers. (Though when replaying a CQ feed it
might be useful to compare the responses to the CR feed.)

If anything like this does not exist, I might write it myself.

I have not used protobufs before so I'm keen to hear advice from those who
have already got their hands dirty / fingers burned.

I'm tempted to weld libfstrm to Lua, so you can configure filtering,
replication, and output with a bit of Lua. The number of Lua protobuf
implementations is a bit of a worry - if anyone has a recommendation I'd
like to short-cut the experimental stage. (I should ask this on the Lua
list I guess!)

Alternatively it might be easier to hack around with the golang-dnstap
code, tho then I would have to think harder about how to configure it...

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at/
justice and liberty cannot be confined by national boundaries