[ratelimits] false positive text

[Vernon Schryver]

> From: Matthijs Mekking <matthijs at nlnetlabs.nl>

> To avoid confusion about what a false/true positive/negative is, it 
> might be good to write down the definitions within the context of RRL. 
> This is a first stab.

In spam filter discussions, I have seen many people concentrate on
the connotations of "false", "positive", "true", "false" and "negative"
while ignoring the long established technical English definitions.
Depending on whether they like or dislike spam filtering in general or
a particular spam filter, they talk about "true" or "false" and "negative"
or "positive" regardless of the standard technical definitions.

Because they consider "false positive" a mortal insult, many other
people have persistently claimed that spam filters based on lists of
SMTP client IP address (mail senders) have no false positives because
all of the entries in the relevant DNSBL were intentional.

It would be best to avoid using of any of the four terms in DNS rate
limiting documents, bug explicitly define them in every document that
does use them.


> A false positive is a packet that is identified as an attack query, when 
> really the packet is legitimate. The legitimate user does not get a 
> response, or it gets a truncated response.
>
> A true positive is a packet that is correctly identified as an attack 
> query. Again, the response is dropped, or a truncated response is sent 
> to the victim.
>
> A false negative is a packet that is identified as a legitimate query, 
> when really the packet is an attack query. The (amplified) response is 
> being reflected to the victim. The first queries at the start of an 
> attack will most likely always be false negatives.
>
> A true negative is a packet that is correctly identified as a legitimate 
> query. The response is sent to the legitimate user.

I agree with those definitions of "true/false positive/negative," but
somewhat disagree about the responses.  What a system does as the
result of a true/false positive/negative test could differ.  For
example, responses to a very small stream of true positives might be
to  respond.  The intentional response to some true negative might be
dropping in the expectation that the client will restransmit and give
so give clues about IDs, RTTs or other characteristics that would
improve later tests.  This is similar to current controversies in
medicine.  Some experts now say that that the response to many true
positives in prostate and breast cancer screen should be no treatment
and not even other tests such as biopsies.

I'm not advocating paradoxical rate-limiting responses, but trying
to distinguish terminology (true/false positive/negative) from policy
(responded/drop/slip).


Vernon Schryver    vjs at rhyolite.com