Don't temp-fail on random address check.

If the address has successfully been checked,
but the subsequent check of a random address
is failing with a 4xx response (or a network
problem pp.), the address has been validated
and should be handled that way.

Signed-off-by: Thomas Hochstein <thh@inter.net>