[dnstap] suggested optional fields for DNSTAP
Joseph Gersch
joe.gersch at secure64.com
Tue Mar 10 20:16:50 UTC 2015
Hi again, Robert,
Here are some comments to your earlier response.
- Joe
> On Feb 27, 2015, at 3:52 PM, Robert Edmonds <edmonds at mycre.ws> wrote:
>
> Joseph Gersch wrote:
>> Hello all,
>>
>> I would like to suggest two optional fields for the DNSTAP schema.
>>
>> The first one has already been discussed, but I don’t see it in the schema yet: a boolean for CACHE-HIT/CACHE-MISS.
>
> Hi, Joe:
>
> Yes, adding a new field to indicate cache status is pretty easy. I held
> back from adding it to the schema just yet because I wanted to get
> feedback from implementers / operators on whether it should be a boolean
> vs. an enum.
>
> In the protobuf wire encoding scheme, booleans and enums use the same
> "variable-length integer" primitive:
>
> https://developers.google.com/protocol-buffers/docs/encoding#structure
>
> So from an efficiency perspective, there's no downside to upgrading the
> field to an enum and encoding a "richer" set of information; they
> consume the same number of bytes on the wire. In particular, I notice
> Google says that their Public DNS service generates the following in
> their "permanent logs":
>
> https://developers.google.com/speed/public-dns/privacy
>
> [...]
> Finally, if you're interested in knowing what else we log when you
> use Google Public DNS, here is the full list of items that are
> included in our permanent logs:
> [...]
> - Whether the request hit our frontend cache
> - Whether the request hit a cache elsewhere in the system (but
> not in the frontend)
> [...]
>
> So, that's two different types of cache hits.
>
> I know that Unbound has both a "message cache" and an "RRset cache", and
> IIRC PowerDNS Recursor also has a "packet cache" which I believe may be
> equivalent to Unbound's message cache. (I think BIND 10 also
> implemented, or at least discussed implementing, a PowerDNS-style packet
> cache, too.) So maybe it makes sense to distinguish between different
> types of cache hits, if there's some consensus between implementations
> on cache types.
>
> What do you think about adding a new enum type to the dnstap.proto
> definitions:
>
> enum CacheStatus {
> // The response message could not be generated entirely from
> // cached information.
> CACHE_MISS = 1;
>
> // The response message was generated by consulting a
> // whole-message cache of DNS responses.
> CACHE_HIT_MESSAGE = 2;
>
> // The response message was generated entirely by consulting a
> // cache of DNS records.
> CACHE_HIT_RECORD = 3;
> }
>
> and then adding a new optional field of that type to the "Message" type:
>
> message Message {
> // [...elided...]
>
> optional CacheStatus cache_status = 15;
> }
>
> This would allow extending CacheStatus to new types of cache hits in the
> future, too.
You give a strong argument as to why it should be ENUM instead of BOOLEAN, and I agree with you.
>
>> The second one is to generate a unique GUID for and store it for each CLIENT_QUERY. This GUID would also be stored with each RESOLVER_QUERY and RESOLVER_RESPONSE. This would allow an analysis of a DNS TRACE to determine operational issues with long recursive resolutions. It is insufficient to just have bailiwick or domain name, because once the recursive resolution starts chasing a CNAME or chain of NS delegations, the domain name changes. Some recursions can take 10-70 lookups to get full resolution. Having a GUID to tie them all together would be very useful.
>
> This is a great use case, but it sounds like it might be a bit hard to
> implement, at least in the recursive DNS server I'm most familiar with
> (Unbound).
I don’t think you should limit the schema based on what’s easy or hard. Some resolvers will be able to do this. And since the field is optional, it won’t add any overhead to the byte stream if a particular DNS Server implementation decides to not do it. But in general, if a DNS Server can do it, this would be very useful information.
If not specified in the schema, I would have two choices: use the EXTRA field, or extend the schema myself to add the additional field. I’m nervous about this latter approach. I don’t see an easy way for multiple vendors to extend the schema without stepping on each other. For example, if I add a field #17 and say it is to be used as a proprietary extension for some purpose, and another vendor also decides to extend the schema and use field #17, then a mess will ensue. I don’t think there is a way for multiple vendors to do independent extensions like they do with OID’s in SNMP. So I would prefer that the first official version of the schema contain fields that are general, and we have an agreement method for extensions. And since all fields are optional, it won’t hurt any particular DNS SERVER to simply choose not to implement particular fields.
>
> Aside: GUIDs/UUIDs specifically would probably be a bit problematic
> compared to just a simple 128-bit random nonce from a seeded CSPRNG
> (which would be more convenient, since recursive DNS servers already
> have those built-in to securely generate random source ports and IDs),
> but that's a relatively minor issue.
Yes, a long random number would be easier and faster.
>
> The big problem, though, is that a single "query tag" is probably not
> enough in the general case. IIUC, the mitigation for VU#457875
> ("Various DNS service implementations generate multiple simultaneous
> queries for the same resource record") [0] in most recursive DNS
> implementations was to aggressively combine outbound queries for the
> same record.
>
> That is, there may not be a mapping only from
>
> RESOLVER_RESPONSE -> CLIENT_QUERY
>
> it's potentially a mapping to a set of multiple unrelated outstanding
> CLIENT_QUERYs:
>
> RESOLVER_RESPONSE -> { CLIENT_QUERY, CLIENT_QUERY, CLIENT_QUERY }
My suggestion is that only the first CLIENT QUERY gets mapped to the “query tag”. The subsequent queries are usually put on a “pending list” and they get a response sent when the original 1st query gets an answer. So a series of fast queries for xyz.com <http://xyz.com/> (and I’ve seen client queries come in immediately on the same IP address and port and even the same transaction ID!!) would look like this:
CQ xyz.com <http://xyz.com/> (query tag 1)
CQ xyz.com <http://xyz.com/> (query tag 2)
CQ xyz.com <http://xyz.com/> (query tag 3)
RQ xyz.com <http://xyz.com/> (query tag 1)
RR xyz.com <http://xyz.com/> (query tag 1)
RQ xyz.com <http://xyz.com/> (query tag 1)
RR xyz.com <http://xyz.com/> (query tag 1)
CR xyz.com <http://xyz.com/> (query tag 1)
CR xyz.com <http://xyz.com/> (query tag 2)
CR xyz.com <http://xyz.com/> (query tag 3)
In other words, each client query generates its own query tag which rides along until that query is finally answered.
>
> I'm not sure if there are limits imposed on how large that set of
> CLIENT_QUERYs might be in recursive DNS implementations, so, in the
> general case, we might need to store a very large vector of query tags
> in the RESOLVER_RESPONSE, and we'd generally prefer to have a hard bound
> on the maximum size of a dnstap protobuf payload. (E.g., given a
> maximum 64K DNS message size plus more than enough room for dnstap
> metadata, it might be reasonable to bound the overall dnstap payload
> length to around ~70 KB, and certainly no more than 128K. Protobuf
> implementations generally decode the entire payload at once into a
> complete in-memory object, so for security reasons it's important to
> bound the size of a protobuf message. Though there are some protobuf
> implementations that can do incremental parsing.)
No, you don’t need a lot of tags, just one per new client query. BTW, I have seen client queries that end up in over a hundred resolver queries. Each one of these would be tagged with just one “query tag” associated with the original client query.
>
> The more practical reason to avoid something like this is that it
> probably requires some intrusive changes to existing recursive DNS
> implementations. (For the Unbound dnstap work, I tried to minimize the
> intrusiveness of the changes required in order to increase the
> likelihood of having the changes accepted by the upstream developer.)
>
> IIUC, Unbound in particular has a fairly opaque callback-based interface
> between what it calls the "serviced query", and the original client
> query; search for "service_callback" in Unbound's
> services/outside_network.h file for details. I'm afraid that would mean
> making some invasive modifications to Unbound's internal API in order to
> gather the "query tag" vector when serializing RESOLVER_RESPONSEs.
>
> Now, it's true that one could map RESOLVER_*QUERY* payloads (rather than
> RESOLVER_RESPONSEs) to exactly the one CLIENT_QUERY that set it off in
> the first place (modulo trivial corner cases like pre-fetching), but
> that solution is a little incomplete without tagging RESOLVER_RESPONSES
> with the full CLIENT_QUERY tag vector; CLIENT_QUERYs that get ganged on
> to an already open upstream fetch will possibly have "dangling" query
> tags (i.e., no corresponding RESOLVER_* payloads with the same tag, or
> an incomplete set of payloads). And those are probably the queries
> you're most interested in analyzing, too.
>
> That is, summing up, I think this is a very nice use case, but it's
> probably pretty hard to implement a really satisfying general solution.
> I'd be happy to be proven wrong, though!
Again, don’t get stuck on whether it’s hard or not. There are other DNS resolvers besides BIND and UNBOUND.
>
> BTW, have you looked at Casey Deccio's dnsviz tool [1] ? Not the
> dnsviz.net service, but the Python tool that drives it. I wonder if
> there might be an alternate approach to the problem of analyzing the
> root-cause of resolution failures that involves active probing, that
> could be triggered by a hypothetical new "timeout" dnstap payload.
> (This is a lot easier, we just need to enumerate and define the
> different kinds of timeouts that can occur.)
I know DNSVIZ (as well as Casey), but I have never looked at the Python tool.
Now, regardless of whether you think these ideas are good or not, I think the base implementation of DNSTAP is already awesome and useful in more ways than we can imagine. I have some interesting HEAT MAPS that I’ve generated. These correlate the domain name to an ontological data base to categorize the types of queries being made by a user base (shopping, searching, travel, news, ……into multiple sub-levels). This and other types of BIG DATA analyses can now go real-time. Yay!
>
> [0] http://www.kb.cert.org/vuls/id/457875
>
> [1] https://github.com/dnsviz/dnsviz
>
> --
> Robert Edmonds
> _______________________________________________
> dnstap mailing list
> dnstap at lists.redbarn.org
> http://lists.redbarn.org/mailman/listinfo/dnstap
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.redbarn.org/pipermail/dnstap/attachments/20150310/4298728b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4124 bytes
Desc: not available
URL: <http://lists.redbarn.org/pipermail/dnstap/attachments/20150310/4298728b/attachment.bin>
More information about the dnstap
mailing list