You can use approach described in the article How to handle improper parameter values in ASP.NET in VB.NET to create a blocking engine to handle and improve parameter values. You should manage invalid requests and notify the client about any invalid parameters that were passed in. You should reply to the request using one of the specific error codes, when someone sends an invalid parameter. You leverage HTTP status codes to maintain compatibility with intelligent clients and to enable forensic log analysis if you need it. Following this approach you can deal with search engines spiders.
The next table lists the principal HTTP error status codes. Each request produces a status code; if there are no errors, the default value is 200 OK:
HTTP status code
Bad request: Used to notify the browser that the request isn’t considered valid.
Not found: The requested content isn’t available.
Error: The request caused an error.
Every request could, in fact, be logged (via IIS) in the corresponding log files and you can use a free tool named Microsoft’s LogParser to analyze Microsoft’s IIS log files. This tool uses a special version of a SQL dialect to submit queries against log files, retrieve the corresponding results, and put those results into different destinations, such as CSV files or a database.
The next code lines show how you can use the 400 bad request HTTP status code when a parameter that is outside the scope is used. You can get this functionality by throwing a new exception of type HttpException:
If (String.IsNullOrEmpty(Request(“ID”))) ThenThrow New HttpException(400, “Bad request”)End If
You can send a detailed response to the client, by changing web.config settings to generate the response from the specific page, as shown:
<configuration><system.web><customErrors mode=”On” defaultRedirect=”GenericErrorPage.htm” redirectMode=”ResponseRewrite”><error statusCode=”400″ redirect=”BadRequest.htm” /><error statusCode=”404″ redirect=”FileNotFound.htm” /></customErrors></system.web></configuration>
The next picture shows the result. You can personalize the look and feel of the page and provide some guidelines for your users:
This approach is useful and you can achieve two results with it:
– You’re notifying the client that there is a problem with the request.
– Your application is string the details in its log files and you can automate collecting and block unwanted IP addresses from doing additional requests if they exceed your threshold.