[dnstap] Python sample to read DNSTAP log or from UNIX socket

[joff.thyer at cybercovenant.com]

Chris,

Having read a little, I have a quick POC Python script to read through and identify Control versus Data frames in the stream.   Before parsing with protobuf, would the idea be to assemble all of the data frames into one large buffer and then parse with “ParseFromString()” or intuitively I would have thought one data frame at a time.

Joff
On May 29, 2019, 4:49 PM -0400, Chris Mikkelson <cmikk at fsi.io>, wrote:
> On Wed, May 29, 2019 at 08:53:19AM -0400, joff.thyer at cybercovenant.com wrote:
> > Hello all,
> > Having recently enabled DNSTAP on my local name server, I was wondering
> > if there was any available Python script to read the binary log
> > structure?
> > I managed to use “protoc” to create the Python module from the
> > “dnstap.proto” definition as follows:
> > protoc --python_out=. dnstap.proto
> > Now I have “dnstap_pb2.py” which I can import into Python3 as follows…
> > but when I tried to “ParseFromString()”, I get an error message.
> > Please note that “dnstap.log” below was produced by “fstrm_capture”.
> > My questions:
> > (1) Is there another way to approach this?
> > (2) Should I consider reading directly from unix socket instead?
>
> The file saved by "fstrm_capture" has framing information in addition
> to the protobuf structure(s). The framing protocol (and, in the case
> of files, file format) is Frame Streams, with implementations in C and
> go at:
>
> https://github.com/farsightsec/fstrm (C)
> https://github.com/farsightsec/golang-framestream (go)
>
> but I know of no python implementation, yet. The protocol is reasonably
> simple, and documented at:
>
> http://farsightsec.github.io/fstrm/group__fstrm__control.html
>
> and should be straightforward to implement (either fully or a useful
> subset) in python.
>
> Thanks,
> --
> Chris Mikkelson
> Farsight Security, Inc.
> cmikk at fsi.io
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.redbarn.org/pipermail/dnstap/attachments/20190529/1ade7027/attachment.htm>