[dnstap] dnstap fanout and replay

[Tony Finch]

> On 13 Apr 2018, at 22:27, Robert Edmonds <edmonds at mycre.ws> wrote:
>  
> The component doesn't even need to be on the same machine as the DNS server if you're using the 'next' branch of fstrm, which has TCP support, though BIND would probably need to be updated to allow configuring an fstrm TCP writer.


Sweet, I should have a look at that!

> This is definitely one of the use cases I had in mind as something that dnstap could support, with the right glue utilities, but nowadays I wonder if the "keeping a standby cache hot" use case wouldn't best be served by existing functionality in dnsdist, if you're using dnsdist to front your recursive servers?

We aren’t - dnsdist is undeniably cool, but I am a bit reluctant to have two layers of health checks / failover when one layer does the job, and we don’t have the amount of traffic or level of abuse that would make it compelling. I quite like being able to implement an optional extra like this off the critical path, so any breakage is decoupled from the main job of serving queries.

> The only advice I can offer would be to watch out for Protocol Buffers v2 versus Protocol Buffers v3 compatibility issues. dnstap was designed using the Protocol Buffers v2 language, and v3 removes some v2 functionality that the dnstap schema uses.

I think the more interesting Lua implementations are plugins to protoc, which I hope means they will work OK!

> One of the cool things about Frame Streams is that it's strongly layered to the point that it doesn't care what is actually in the payloads that it transports (other than carrying a "content type" for the user to describe the type of payloads carried in the stream), so if you don't actually need to edit or consume the dnstap protobuf payloads, you could certainly write a fanout utility in Lua without needing a protobuf dependency at all. Though for replaying, you would certainly need to deserialize each payload to extract the query messages.

Oh, right, I was vaguely aware of this layering but I thought the client/resolver/etc. query/response tag was in the protobuf blob rather than the fstrm metadata (it’s described in the dnstap.proto file) which would imply even a simple dnstap filter needs to know about protobuf...

Thanks for the tips about code to look at!

Tony.
-- 
f.anthony.n.finch  <dot at dotat.at>  http://dotat.at


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.redbarn.org/pipermail/dnstap/attachments/20180415/4d95f74c/attachment-0001.html>