Skip to main content
added 43 characters in body
Source Link
Marcus Müller
  • 6.3k
  • 2
  • 20
  • 29

You're hurting yourself.

The "attacker"/crawler... probably doesn't pay for their traffic or processing power (ie. they use a botnet, hijacked servers or at least are on a connection that doesn't make them pay for traffic), but you will be billed for traffic and CPU/storage/memory, or your server's hoster has a "fair usage" clause, under which your server's connection will be throttled or cut off if you serve Gigabytes of data on the short term, or your storage bandwidth will be reduced, or your CPU usage limited.

Also, why should anyone be so stupid to download Gigabytes of data when they're just looking for a specific page? Either, they're just looking for existence of that page, in which case the page's size won't matter, or they will definitely set both a timeout and a maximum size – no use waiting seconds for a server to complete a response if you've got hundreds of other servers to scan, and especially not when greylisting is a well-known technology to slow down attackers.

You're hurting yourself.

The "attacker"/crawler... probably doesn't pay for their traffic (ie. they use a botnet, hijacked servers or at least are on a connection that doesn't make them pay for traffic), but you will be billed for traffic, or your server's hoster has a "fair usage" clause, under which your server's connection will be throttled or cut off if you serve Gigabytes of data on the short term.

Also, why should anyone be so stupid to download Gigabytes of data when they're just looking for a specific page? Either, they're just looking for existence of that page, in which case the page's size won't matter, or they will definitely set both a timeout and a maximum size – no use waiting seconds for a server to complete a response if you've got hundreds of other servers to scan, and especially not when greylisting is a well-known technology to slow down attackers.

You're hurting yourself.

The "attacker"/crawler... probably doesn't pay for their traffic or processing power (ie. they use a botnet, hijacked servers or at least are on a connection that doesn't make them pay for traffic), but you will be billed for traffic and CPU/storage/memory, or your server's hoster has a "fair usage" clause, under which your server's connection will be throttled or cut off if you serve Gigabytes of data on the short term, or your storage bandwidth will be reduced, or your CPU usage limited.

Also, why should anyone be so stupid to download Gigabytes of data when they're just looking for a specific page? Either, they're just looking for existence of that page, in which case the page's size won't matter, or they will definitely set both a timeout and a maximum size – no use waiting seconds for a server to complete a response if you've got hundreds of other servers to scan, and especially not when greylisting is a well-known technology to slow down attackers.

Source Link
Marcus Müller
  • 6.3k
  • 2
  • 20
  • 29

You're hurting yourself.

The "attacker"/crawler... probably doesn't pay for their traffic (ie. they use a botnet, hijacked servers or at least are on a connection that doesn't make them pay for traffic), but you will be billed for traffic, or your server's hoster has a "fair usage" clause, under which your server's connection will be throttled or cut off if you serve Gigabytes of data on the short term.

Also, why should anyone be so stupid to download Gigabytes of data when they're just looking for a specific page? Either, they're just looking for existence of that page, in which case the page's size won't matter, or they will definitely set both a timeout and a maximum size – no use waiting seconds for a server to complete a response if you've got hundreds of other servers to scan, and especially not when greylisting is a well-known technology to slow down attackers.