[ratelimits] Multiple buckets

[paul vixie]

On 10/25/2012 11:54 PM, Jay Daley wrote:
> Hi
>
> When I've previously implemented rate limiting systems I've found that miscreants get to learn the limits and adjust accordingly.  For example, if I wished to launch a reflection attack then I would not be bothered by a server running a rate limit of 10qps, I would just find 1000 such servers and use all of them at once.  With the current RRL implementation I could happily run this attack for hours or even days.

so, i get it. bad guys can run at N-1 QPS split across M servers and get
(N-1)*M RPS to the victim. and i can see why {N-1,N+1,N-1,N+1} would be
the observable edge effect at the amplifier. but as vernon already
explained, that's not what's happening here, since our "stopped" message
isn't on time, it's 60 seconds later. this means your idea for multiple
token buckets...

> I would recommend that the RRL implementation be extended to provide multiple, configurable token buckets.  

...is not strictly called for by the evidence at hand, though it has the
sound of "necessary". what we don't want to do though is keep that much
state-memory around. in your example there was an hour-long window.
that's hugely more heap and hash than we do now, by many orders of
magnitude. i'm not saying it can't be done or that it definitively does
not need doing, but i will say, let's wait on it for now.

> Another benefit of a more general approach like this is that everybody does it differently, which makes it far harder for miscreants to predict cumulative behaviour and use cumulative behaviour to their advantage.

miscreants will, alas, map their resources and remember "N" for each
reflector, possibly in a spreadsheet. they are tool using creatures and
they have skills similar to ours but with far greater incentives than ours.

paul