From mail at kfiducia.com Fri May 12 19:21:38 2017 From: mail at kfiducia.com (Kyle Fiducia) Date: Fri, 12 May 2017 15:21:38 -0400 Subject: [dnstap] dnstap use case assistance Message-ID: There aren't a lot of folks using dnstap from whom to ask for help, so I am hoping you folks can help educate me! Use Case (Threat Detection & Response): If a packet gets flagged for malicious content, it is flagged by source IP: lets say "1.2.3.4" We investigate by asking "how did the user get to 1.2.3.4?" - Generally this is by DNS reference (evil-host.tld), which is what brought me to dnstap. With dnstap/unbound doing all the dns resolutions on the network I attempt to cross references the resolutions that yield 1.2.3.4 back to domain names to help trace the source of the potential compromise (perhaps this domain name is included in an email or something). But the logs I am seeing coming out of DNS tap reference the responding authoritative server. So it mentions "evil-host.tld" but gives the authoritative nameserver IP address that responded with the A record, not the contents of the A record, so 1.2.3.4 is not present in my logs. Realizing I may be using dnstap in a fashion it wasn't intended, I hope someone can steer me towards a more viable solution to improve my Incident Response capability, it seems this is well within the wheelhouse of such a useful tool, perhaps I am just missing something. Thank you in advance for any time and attention you can offer. Best, Kyle My current dnstap config (trying to get as much as I could out of it to ensure I wasn't missing something): dnstap: dnstap-enable: yes dnstap-socket-path: "/var/run/unbound/dnstap.sock" dnstap-send-identity: yes dnstap-send-version: yes dnstap-log-client-query-messages: yes dnstap-log-client-response-messages: yes dnstap-log-resolver-query-messages: yes dnstap-log-resolver-response-messages: yes dnstap-log-forwarder-query-messages: yes dnstap-log-forwarder-response-messages: yes From paul at redbarn.org Fri May 12 23:49:30 2017 From: paul at redbarn.org (Paul Vixie) Date: Fri, 12 May 2017 16:49:30 -0700 Subject: [dnstap] dnstap use case assistance In-Reply-To: References: Message-ID: <59164A0A.8070802@redbarn.org> dnstap is better, faster, transaction logging. it can be compared to syslog or querylog. how you postprocess the results isn't covered. the contents of the resource records you look up *is* present in the output, according to the unbound config you gave. -- P Vixie From edmonds at mycre.ws Sat May 13 04:19:20 2017 From: edmonds at mycre.ws (Robert Edmonds) Date: Sat, 13 May 2017 00:19:20 -0400 Subject: [dnstap] dnstap use case assistance In-Reply-To: References: Message-ID: <20170513041920.okbwn2n26z7x47kh@mycre.ws> Kyle Fiducia wrote: > But the logs I am seeing coming out of DNS tap reference the > responding authoritative server. So it mentions "evil-host.tld" but > gives the authoritative nameserver IP address that responded with the > A record, not the contents of the A record, so 1.2.3.4 is not present > in my logs. Hi, Kyle: The data you're looking for is going to be included in the DNS answer section of the DNS response message payload. And there might be more than one A record for "evil-host.tld" in the response, of course. What tool are you using to decode your dnstap logs? The three dedicated tools that I'm aware of (the standalone dnstap-ldns and golang-dnstap tools, and the dnstap-read tool included with BIND) default to using a compact one line per message representation that resembles what you described above, but they also have a more verbose format that can be enabled with the '-y' parameter which includes a full decode of the DNS response sections. BTW, at one point I was working on adding support for decoding Frame Streams files with dnstap-encoded payloads to Wireshark. I started it but didn't finish it; the hardest part looked like it was going to be integrating a protobuf decoder and code generator into the build system (and making it work on the platforms supported by Wireshark), rather than actually implementing the dnstap dissector. -- Robert Edmonds From mail at kfiducia.com Sat May 13 16:39:52 2017 From: mail at kfiducia.com (Kyle Fiducia) Date: Sat, 13 May 2017 12:39:52 -0400 Subject: [dnstap] dnstap use case assistance In-Reply-To: <20170513041920.okbwn2n26z7x47kh@mycre.ws> References: <20170513041920.okbwn2n26z7x47kh@mycre.ws> Message-ID: Paul, Robert, Let me first say thank you to both of your for your immense contributions to the space, and for taking the time to push me in the right direction! Its humbling to have people that have made such significant contributions offer assistance. I am using the golang-dnstap tool to read the socket, and after Paul's assurance the data was there, I educated myself further on how to get more verbose data out of the stream with golang-dnstap. I was previously just dumping to stdout, incorrectly assuming it would offer what I was looking for and found what Robert just explained: use the -y. As Paul promised, the data is there, and accomplishes my goal. My next task is finding an efficient way to parse the answer section, store, and query this data. For now, I am pleased to have a such a useful and cleverly designed tool to work with. Thanks to you both! Kind Regards, Kyle On Sat, May 13, 2017 at 12:19 AM, Robert Edmonds wrote: > Kyle Fiducia wrote: >> But the logs I am seeing coming out of DNS tap reference the >> responding authoritative server. So it mentions "evil-host.tld" but >> gives the authoritative nameserver IP address that responded with the >> A record, not the contents of the A record, so 1.2.3.4 is not present >> in my logs. > > Hi, Kyle: > > The data you're looking for is going to be included in the DNS answer > section of the DNS response message payload. And there might be more > than one A record for "evil-host.tld" in the response, of course. > > What tool are you using to decode your dnstap logs? The three dedicated > tools that I'm aware of (the standalone dnstap-ldns and golang-dnstap > tools, and the dnstap-read tool included with BIND) default to using a > compact one line per message representation that resembles what you > described above, but they also have a more verbose format that can be > enabled with the '-y' parameter which includes a full decode of the DNS > response sections. > > BTW, at one point I was working on adding support for decoding Frame > Streams files with dnstap-encoded payloads to Wireshark. I started it > but didn't finish it; the hardest part looked like it was going to be > integrating a protobuf decoder and code generator into the build system > (and making it work on the platforms supported by Wireshark), rather > than actually implementing the dnstap dissector. > > -- > Robert Edmonds