[ratelimits] BIND9 DNS rate limit logging

[Patrik Fältström]

On 7 jul 2012, at 14:47, Vernon Schryver wrote:

>> From: =?iso-8859-1?Q?Patrik_F=E4ltstr=F6m?= <paf at frobbit.se>
> 
>> One thing I have been looking for is "counting" both the number
>> of blocked queries and the number of intervals where blocking occurs.
>> And today we get the number of "windows", as we get the log entry
>> when the window "opens" (i.e. when queries starts to get dropped
>> in that window). You suggest getting data in the query category for
>> also dropped queries which is good, but, it feels complicated to
>> be forced to save the query category just to do the counting.
>> 
>> Could one also get a log entry when a window is opened again (i.e.
>> when the blocking for that window stops) where also the cumulative
>> number of dropped queries according to that window rule has been counted?
> 
> Counters seem reasonable and possible, but I'm not sure how many there
> should be.  I think these events might be counted:
>   - dropped ordinary response
>   - dropped NXDOMAIN response
>   - dropped other error response
>   - 'slipped' (TC=1) ordinary response
>   - 'slipped' (TC=1) NXDOMAIN response
> Do you want 5 counters, one for each type of action, 1 counter 
> for all of them together, or an intermediate number of counters?
> It would be easiest to have 1 counter for everything, or perhaps 2
> counters, 1 for dropped other error responses and a second for
> everything else.

If by "dropped" you imply "something that hit the a rate limit window", I think that is the counter I am asking for.

> Logging when a window is opened again sounds interesting, but
> expensive.

Not when it is opened again, but when it closed. When you do some garbage collect or something. At that time I want to know how many in total has hit that rule that was dynamically created.

Today, if I understand correctly, you log in the rate limit class once first time a query hit a dynamically created rule (a bucket), i.e. first time a query is dropped?

> The rate limiting machinery is triggered only by requests
> and responses.  There are no timers and nothing happens in their
> absence.

How do you remove buckets?

For example, I have the following:

rate-limit {
  responses-per-second 5;
  IPv4-prefix-length 8;
  window 200;
  slip 10;
};

Are you logging as rate limit _every_ query that is dropped due to this rule today?

> Doing something such as logging when a window opens would
> require waking up spontaneously and examining the entire rate limiting
> database.

Understood.

> Another complication is ensuring that a log entry is made
> when an active bucket is stolen or recycled.  Hidden complications
> include locking, shutting down and starting the timer upon `rndc reload`.
> It wouldn't be hard, but ...

Hmm...yes.

> I propose to change the code to log all drops or slips (TC=1) in the
> "queries" category and the start of dropping of a stream of responses
> in the "rate-limit" category.

Ok.

> It repeats the "rate-limit" log entry
> once every 60 seconds while the dropping of that stream continues so
> that a continuing attack is not missed as log files are rotated.

Is it possible to also get in each such log entry "number of hits" the last 60 seconds?

> Is 60 seconds too often or not often enough?

I think it is ok.

> Would the ending of 60 (?) second still rate limit messages be good enough
> to mark the opening of a window?

For me, yes.

I.e. for me it is enough to log once every 60 seconds per bucket that have been in use (regardless of whether it has been created or not), but I would like to get the number of hits as well.

   Patrik