[dnstap] dnstap fanout and replay

[Robert Edmonds]

Paul Vixie wrote:
> Jerry Lundström wrote:
> > 
> > I would gladly see dnstap move towards CBOR, it's basically the same thing
> > as msgpack/protobuf so the transition should be easy.
> > 
> 
> cbor is pretty recent. does it have a code generator for C yet?

RFC 7049 is from 2013, which was five years ago.

The initial public release of Protocol Buffers was in 2008, and the
first commits in the dnstap.pb repository are from 2013. So when work on
dnstap began, protobuf was about as old as CBOR is now.

> dnstap is now years old. i don't expect it to change its on-wire format.

Byte stream stability is a very defensible argument. But, future
implementations would have to deal with the technical debt of dnstap
being implemented in the protobuf 2 language, and the protobuf
ecosystem, while having a significant amount of open source technology
is still basically controlled by Google. E.g., except for a few niche
implementations, almost everybody uses Google's protobuf C++ compiler
library or implements their code generator as a plugin for Google's
protoc tool, *including* the protobuf-c implementation that the DNS
servers written in C have used for their dnstap implementations.

The dnstap protobuf schema is written in the proto2 language and uses
the 'optional' keyword heavily, which has been removed from proto3. As
long as new versions of Google protobuf continue to implement support
for proto2, the techical debt of dnstap being implemented in proto2 is
fairly manageable. There is a risk that Google will decide to retire
proto2 support from new versions of protobuf in the future, at which
point the dnstap ecosystem would be in the awkward position of telling
users that they need to build software against older releases of
protobuf. Or it may be possible to port the dnstap schema from proto2 to
proto3.

Note that I'm not advocating for or against a breaking format change for
dnstap (I don't use it enough these days to really have a say), but if a
new format is needed, it can easily support a superset of the semantics
of the previous format, and a tool to convert old captures is easy
enough to develop.

-- 
Robert Edmonds