[dnstap] dnstap fanout and replay

[Robert Edmonds]

Jerry Lundström wrote:
> I would gladly see dnstap move towards CBOR, it's basically the same thing
> as msgpack/protobuf so the transition should be easy.

There are major differences among CBOR, msgpack, and protobuf. It
doesn't really make sense to refer to them all as basically the same
thing.

protobufs are not self describing and require a schema to define the
data types and field names of particular wire fields. There isn't a 1:1
mapping of wire types to data types, for instance "varints" on the wire
can be int32, int64, uint32, uint64, etc.

MessagePack is sort of a JSON inspired binary encoding. I think the only
reason it's interesting for this mailing list is because it can store
arbitrary binary data (e.g., DNS wire messages) efficiently. Otherwise
you get many of the downsides of JSON plus the ecosystem downside of
being much more obscure than JSON.

The thing that really distinguishes CBOR from protobuf for me is the
IANA "Concise Binary Object Representation (CBOR) Tags" registry
(https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml). There is
nothing like this for protobuf. This is sort of like the RRTYPEs
registry for DNS. If I understand correctly, CBOR lets you apply a "tag"
to a field on the wire that describes the semantics of the field.
protobuf, on the other hand, doesn't have any space for these extra
semantics and they have to be carried around in programmer's heads, or
in comments. E.g., take this field in the dnstap protobuf schema:

https://github.com/dnstap/dnstap.pb/blob/2d8098aaef53e548e3808f757e48acc51c37b6c9/dnstap.proto#L209-L211

    // The "zone" or "bailiwick" pertaining to the DNS query message.
    // This is a wire-format DNS domain name.
    optional bytes              query_zone = 11;

This field is a protobuf byte string, so it can literally have any
binary data in it, and there's a comment requesting that implementers of
encoders and decoders treat this field as a wire-format DNS name. In
CBOR you could register a tag for a DNS wire-format name and make this
request have an explicit wire-format representation and standardized
meaning. Similarly for other data types like DNS wire messages, IP
addresses, time stamps, etc.

-- 
Robert Edmonds