[DNSfirewalls] rpz firewall + whitelisting

Tue Aug 27 22:04:17 UTC 2019

Hi Vernon,

The Deteque/Spamhaus RPZ zones were refactored several years ago to
limit any one zone from being overlay large.  At the present time, our
largest zone is dga.host.dtq, which contains DGA domains, with just
under 1.1 million entries.  The second largest zone is our
badrep.host.dtq, which contains domains with bad reputations, with just
under 925,000.

The DGA zone is generated every 12 hours.  Our badrep zones are a bit
more active, with updates pushed out every 120 seconds.

To get a better idea of the zone sizes:

      692 adware.edit.host.dtq
      704 adware.host.dtq
     2644 bad-nameservers.host.dtq
      488 bad-nameservers.ip.dtq
   313522 badrep.edit.host.dtq
     5648 badrep.hacked.host.dtq
   924084 badrep.host.dtq
   216259 bogons.ip.dtq
    14662 botnetcc.edit.host.dtq
     2898 botnetcc.hacked.host.dtq
      160 botnetcc.hacked.ip.dtq
    51296 botnetcc.host.dtq
     1185 botnetcc.ip.dtq
     2494 botnet.edit.host.dtq
     3228 botnet.host.dtq
    52734 coinblocker.srv
  1071214 dga.host.dtq
      957 drop.ip.dtq
     2522 malware.edit.host.dtq
     2292 malware.hacked.host.dtq
     2736 malware.host.dtq
    37686 phish.edit.host.dtq
      668 phish.hacked.host.dtq
    48670 phish.host.dtq
  3963164 porn.host.srv
    15813 sbl.ip.dtq
     1061 torblock.srv
   386946 zrd.host.dtq

Andy

On 8/26/19 7:08 PM, Vernon Schryver wrote:
>> From: Brian Dickson <brian.peter.dickson at gmail.com>
>> To: Bob Harold <rharolde at umich.edu>
>> Cc: dnsfirewalls at lists.redbarn.org
> 
>>> If your local list and the third party list are separate RPZ zones, then
>>> it should be almost fine, I think.  Each zone is processed separately, and
>>> the first zone that matches takes effect.  The third party would not match,
>>> but yours would.  I know that sounds confusing, you might want to test it.
>>>
>> The implementation I am familiar with, uses a bit-field (32 bits) for zone
>> membership.
> 
> Would that be my second implementation for ISC in BIND9?  If so,
> the 32-bit business was a speed-up based on assuming that no one would
> want more than 32 policy zones.  My first implementation was
> based on the assumption that 2 or 3 zones would be more than enough
> for anyone.  It involved iteratively checking all policy zones for
> hit, for each of the possible types of hit.  It didn't work so well
> for people who decided they needed a dozen zones, and the 3rd didn't
> work for more than 32 policy zones and had performance problems when
> things needed to be rebuilt.  So it goes.
> 
> The 32-bit masks are in a separate tree/hash and say which, if any, of the
> policy zones might have a hit for domain.
> (Contrary to comments in the source about "red/black trees", BIND9
> uses trees of hash tables, at least when I last looked at the code.)
> 
> My third implementation was the so call FastRPZ for Farsight Securities
> and uses a different scheme that makes it significantly faster for
> very large policy zones such as Spamhaus's and removes the 32 zone
> limit.  There is a separate deamon that maintains a database shared
> with an 'RPZ client' such as BIND or Unbound.  The RPZ client is not
> impeded by the policy zone maintenance work of the separate daemon.
> The database also lives on disk, which makes restarting almost painless.
> There may be a way to get FastRPZ for BIND9 via ISC.
> 
> 
>> And, it uses a well-defined ordering on the zones themselves.
>>
>> So, you can have up to 32 zones in a sequence to be used with a first-found
>> matching, based on the order you use for the zones.
> 
> For a definitive description of the RPZ 'protocol' including the
> precedence of zones and types of policy rules, see
> https://datatracker.ietf.org/doc/draft-ietf-dnsop-dns-rpz/
> or
> https://www.ietf.org/archive/id/draft-ietf-dnsop-dns-rpz-00.txt
> 
> 
>> Examples would be things like:
>>
>>    - absolute whitelist
>>    - high-quality curated blacklist
>>    - high-value whitelist
>>    - high-trust, high-quality blacklist
>>    - internally generated high-quality whitelist
>>    - medium-trust, high-quality blacklist (possibly merged from multiple
>>    sources)
>>    - low-trust whitelist
>>    - low-quality blacklist
>>    - everything else passes by default, or everything else is blocked by
>>    default, whichever makes sense.
>>
>> IIRC there is a modest performance penalty based on the number of zones you
>> use, but not based on the size of the zones themselves except at
>> load/reload time.
> 
> I agree with that, with the caveats that:
> 
>   - the 32-bit business almost entirely fixed the performance penalty of
>      having many zones (up to 32) but at some performance cost when
>      a policy zone must be reloaded.
> 
>   - the cost of (re)loading a very large zone such as some of Spamhaus's,
>      including when the authority server expires the entire previous
>      version instead of sending DNS protocol 'incremental' changes,
>      can be a matter of several minutes(!) of CPU time.
> 
> 
> Vernon Schryver    vjs at rhyolite.com
> _______________________________________________
> DNSfirewalls mailing list
> DNSfirewalls at lists.redbarn.org
> http://lists.redbarn.org/mailman/listinfo/dnsfirewalls
> 

-- 
Andrew Fried
afried at deteque.com

+1.703.667.4050  Office(US)
+44.20.81140020  Office (UK)
+1.703.362.0067  Mobile
deteque          Skype