[RPZ] Logging analysis and consolidation (part II)

[Hugo Maxwell Connery]

Hi,

Since my question about a tool for using RPZ as a security analysis tool
back in January, I have put together the basics that I wished for.  It is very
much still an alpha product, but delivers the core of the data I wished to 
access.

See the attached one page graphic for a picture of what is happening.

In summary:

* all RPZ responses from DNS resolvers are logged and shipped to a database
* all visits to the walled garden are logged and shipped to the same database
* the analysis web site allows querying of the raw data (DNS or Walled Garden), 
  or the more interesting correlated data (DNS log + Walled Garden log --> person 
  visiting nasty sites, or DNS with *no* corresponding Walled Garden log --> probable 
  malware activity).

Raw data is viewable as plain raw, or in aggregated form (total number of 'hits' by
each unique host in a period showing IP address, hostname, queried domain 
and the total).  Consolidated data is always aggregated.

The log scrapers for BIND 9.8.1 and Apache 2 are built, as is the web-site which
provides access to the data.  Runs well on any Linux distro.  A few small changes
will enable any OS that supports Perl and/or Postgres.

If anyone is interested in this, please contact me.  I am particularly interested 
in finding out other peoples wishes for types of data queries, and other manners
in which the mechanism(s) can be improved.

Immediate plans are:

* finalise initial functionality, documentation and installation instructions
* implementation in my networks
* publish tools at an open source hosting environment (probably github.com)
* establishment of an example site with sanitised data

(estimate completion time 4 weeks).

I hope that there are others interested in the use of RPZ as a security analysis tool.

Any interested person's input towards this effort would be gladly appreciated.

If I am misusing this mailing list, please let me know and I will gladly cease and desist.

Regards,
--
Hugo Connery, Head of IT, DTU Environment
http://www.env.dtu.dk

PS: Here is the type of data that one sees for a DNS + Walled Garden query.

date_trunc 	       count 	client_ip 	client_hostname 	query_domain
2012-02-06 00:00:00 	6 	1.2.3.6 	host3.org 	4.nasty.com
2012-02-06 00:00:00 	4 	1.2.3.4 	host1.org 	1.nasty.com
2012-02-06 00:00:00 	2 	1.2.3.5 	host2.org 	2.nasty.com
2012-02-06 00:00:00 	2 	1.2.3.5 	host2.org 	3.nasty.com 

PPS: A recent detailed report into the mechanisms behind the RSA attacks last year (and some other
similar attacks) makes recommendations regarding the configuration of local DNS 
resolvers that seem relevant to RPZ (and the tool being developed).  The report can
be viewed at:

  http://www.commandfive.com/papers/C5_APT_C2InTheFifthDomain.pdf  (PDF 600 KB)

(I have no association with the security company that published the report).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Pictorial Overview.odg
Type: application/vnd.oasis.opendocument.graphics
Size: 14571 bytes
Desc: Pictorial Overview.odg
URL: <http://lists.redbarn.org/pipermail/dnsfirewalls/attachments/20120217/fa2ce98f/attachment.odg>