Installation and deployment Questions on installation, system configuration and deployment to the live server.
Version 5.x > Installation and deployment > Search engine bot takes down the site View modes: 
User avatar
Member
Member
1228 - 4/10/2011 8:14:26 PM
   
Search engine bot takes down the site
Hi,

Every day google, yahoo and bing bots crawl our site and causes a lot of exceptions being throwned. All these exceptions makes the site go down and we have to ask our web host to clear the .Net chache and restart the app pool. The two following erorrs are:

"Event ID: 4187
Event type: Error
Event time: 4/8/2011 8:14:15 PM
Source: Application_Error
Event code: EXCEPTION
User ID: 65
User name: public
IP address: 66.249.67.227

Description: Message: Length of the data to decrypt is invalid.
Stack Trace:
at System.Security.Cryptography.RijndaelManagedTransform.TransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
at System.Security.Cryptography.CryptoStream.FlushFinalBlock()
at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, IVType ivType, Boolean useValidationSymAlgo)
at System.Web.UI.Page.DecryptStringWithIV(String s, IVType ivType)
at System.Web.UI.Page.DecryptString(String s)

Site name: site
Machine name: VENOM
Event URL: /ScriptResource.axd?d=jZJkR9OGWSRKNyKFVUTyoKXK9rlmh5iZl3G7DA6UYeObtbVvSUH_ck9r1QNIIeRxrFihFKBN2ZSx4418TjyPX4FQqsoYbA8VZt6_YUkAGIPFMEOr1bI2WuIxahgqpoBGLN6GDDXfW2dpa47RSaO-_OSudw2euSPyo9HBlOShd9f5y6Hu0&t=ffffffffd44210f8
URL referrer:
User agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"


and

"Event ID: 4233
Event type: Error
Event time: 4/9/2011 5:27:07 AM
Source: Application_Error
Event code: EXCEPTION
User ID: 65
User name: public
IP address: 66.249.67.218

Description: Message: Padding is invalid and cannot be removed.
Stack Trace:
at System.Security.Cryptography.RijndaelManagedTransform.DecryptData(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount, Byte[]& outputBuffer, Int32 outputOffset, PaddingMode paddingMode, Boolean fLast)
at System.Security.Cryptography.RijndaelManagedTransform.TransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
at System.Security.Cryptography.CryptoStream.FlushFinalBlock()
at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, IVType ivType, Boolean useValidationSymAlgo)
at System.Web.UI.Page.DecryptStringWithIV(String s, IVType ivType)
at System.Web.Handlers.AssemblyResourceLoader.System.Web.IHttpHandler.ProcessRequest(HttpContext context)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Site name: site
Machine name: VENOM
Event URL: /WebResource.axd?d=w7Y4NZGuwrbjFk-7D62R-g2&t=634105738304687500
URL referrer: oursite
User agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

How can we get rid of these annoying errors?


Thanks in advance.

User avatar
Kentico Support
Kentico Support
kentico_jurajo - 4/11/2011 10:53:53 AM
   
RE:Search engine bot takes down the site
Hi,

1. are you using web farms? If so, please take a look here http://forums.asp.net/t/1225162.aspx

2. Please see this KB article for more details.

Moreover, I would like to recommend using robots.txt file to exclude the bots from certain files and folders on your web site (like the .axd file or CMSPages folder).

Best regards,
Juraj Ondrus