[ratelimits] CH/TXT/id.server queries rate-limited
vjs at rhyolite.com
Fri Oct 26 06:06:58 UTC 2012
> From: Jay Daley <jay at nzrs.net.nz>
> > The "stop limiting" messaages should only happen when either
> > - 60 seconds after rate limiting stops
> Or are you just talking about the limiting behaviour and not the
> generation of log messages:
> Either you are rate limiting at X qps in which case it should stop
> in the first second where the traffic is < X or you are actually
> describing the use of multiple buckets as I described previously. To
> put it simply, if you stop limiting 60 seconds later then you are
> actually talking about X qpm not X qps.
I also wrote:
} In other words, even if fence-posting is happening, it should not
} be visible in the log messages.
Consider that stream of 11/9/11/9/11/9/11/9/... requests. If you don't
use hysteresis (when possible) on the "stop" messages, you get a "start"
or a "stop limiting" message every second and more complaints about
too many log messages from people running big DNS servers.
(It's not possible if you're not allowed enough buckets to delay
> > } For example, if I wished to launch a reflection attack then I would not
> > } be bothered by a server running a rate limit of 10qps, I would just
> > } find 1000 such servers and use all of them at once. With the current
> > } RRL implementation I could happily run this attack for hours or even
> > } days.
> I wasn't talking about sources, I was talking about servers - multiple
> servers from different operators being harnessed simultaneously.
However, there is no practical number or kind of rate rate limiting
buckets that solves that problem without what I think are intolerable
false positives. For example, while no single simple host is likely
to ask for most RRsets 5000 times in 3600 seconds (e.g. your bucket3),
a block of IP addresses (and especially a block of IP addresses involving
carrier grade NAT) might easily send more than 5000 requests/hour for
a popular domain name.
A separate problem with the idea of rate limiting that can detect
a stream of 5000 responses in 3600 seconds is that you would need
to keep a bucket for every IP address block seen for 3600 seconds.
At 100 qps that is a modest 360K buckets and no more than a few
MByte of RAM. At high query rates, you start needing more RAM,
which gets back to what I suspect is causing this complaint about
many "start" and "stop limiting" log messages.
Vernon Schryver vjs at rhyolite.com
More information about the ratelimits