[dnstap] DNSTAP vs PCAP

[Paul Vixie]

On Friday, February 3, 2017 6:24:53 AM PST Barry Raveendran Greene wrote:
> 
> Thanks Paul. Some conversations I’ve had say that PCAP will work just as well as DNSTAP. What how would we contrast the advantages with the way DNSTAP can be more effectively transported, aggregated, and pulled into the collector?

pcap's general purpose tooling does not easily support streaming output. yes, libpcap and bpf can be used that way, see for example ncap and nmsg from SIE. but ultimately the pcap community use dnscap and tcpdump and wireshark, and we make files, and then we rsync those files.

'dnstap' was designed as a continuous telemetry source. you _can_ make files but that's unusual. 'dnstap' represents a commitment by DNS agent implementers and their operators to continuously monitor the state of their DNS apparatus including both things you can also see on the wire, and things like cache purge events that don't explicitly show up on the wire.

pcap also places a decoding burden on each analyst. a pcap header may have a DLT that the analyst hasn't seen before and won't nec'ily be able to skip over to find the L3 packet. the L3 packet can include IP4 and IP6 headers including extension headers which may not be meaningful but must be understood well enough to skip over. the UDP headers are simpler. but just getting to the DNS payload is per-analyst work that pcap requires and that 'dnstap' does not. then there's the problem of fragment reassembly, reassociation of question and response packets, and handling TCP, all of which are required for high quality analysis results, none of which is universal, and none of which are necessary for a 'dnstap' user.

PCAP inspired NCAP which inspired NMSG which inspired 'dnstap', all because various people wanted to make the gathering, sharing, and analysis of DNS telemetry so easy that it would become universal.

vixie

re:

> 
> 
> > On Feb 2, 2017, at 2:13 PM, Paul Vixie <paul at redbarn.org> wrote:
> > 
> > On Thursday, February 2, 2017 7:56:15 AM PST Barry Raveendran Greene wrote:
> >> 
> >> Has anyone done a good contrast between DNSTAP vs PCAP streaming? DNSTAP is picking up momentum. The FAQ would be how it compares to PCAP.
> > 
> > Whereas PCAP is a low level packet storage and transfer format and
> > associated tools, 'dnstap' is a high-level DNS-specific telemetry
> > storage and transfer format and associated tools. The 'dnstap' format
> > for example does not carry the ISO-L2 ("ethernet") addresses associated
> > with queries and responses, and it can associate a query with its
> > response and store or transfer the resulting transaction as a single
> > atomic unit. Finally, since 'dnstap' resides in the DNS protocol agent
> > (client, server, or proxy) it can carry information that would never
> > otherwise appear "on the wire" outside of the DNS protocol agent. For
> > example, the "working delegation-point" of a transaction reported by
> > 'dnstap' can be reported explicitly, whereas for a transaction whose
> > packets are witnessed via PCAP, the "working delegation-point" must be
> > imputed/guessed.

-- 
P Vixie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.redbarn.org/pipermail/dnstap/attachments/20170203/0783d300/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .vcf
Type: text/vcard
Size: 268 bytes
Desc: not available
URL: <http://lists.redbarn.org/pipermail/dnstap/attachments/20170203/0783d300/attachment.bin>