Kentico 10 LogHit errors for web crollers

profound Team asked on January 9, 2018 18:12

Hi,

I have been going through the Application Insights for two different websites we run and noticed several thousands errors recorded all with commonalities. The error is caused when POST is made to webanalytics/LogHit and returns 403, this is only happening when request is coming from IP Address: 66.249.76.0 Name: GOOGLE Handle: NET-66-249-64-0-1 Registration Date: 05/03/04 Range: 66.249.64.0-66.249.95.255 Org: Google LLC Org Handle: GOGL Address: 1600 Amphitheatre Parkway City: Mountain View State/Province: CA Postal Code: 94043 Country: United States

This is suggesting that this request is made by the web crawler and the following IIS error is triggered:

IIS Detailed Error - 403.0 - Forbidden HTTP Error 403.0 - Forbidden You do not have permission to view this directory or page.

Most likely causes: This is a generic 403 error and means the authenticated user is not authorized to view the page.

Things you can try: Create a tracing rule to track failed requests for this HTTP status code. For more information about creating a tracing rule for failed requests, click http://go.microsoft.com/fwlink/?LinkID=66439 here.

Detailed Error Information: Module   ManagedPipelineHandler Notification   ExecuteRequestHandler Handler   System.Web.Http.WebHost.HttpControllerHandler Error Code   0x00000000

Requested URL   https://my-kentico-app__f394:80/cmsapi/webanalytics/LogHit Physical Path   D:\home\site\wwwroot\cmsapi\webanalytics\LogHit Logon Method   Anonymous Logon User   Anonymous

More Information: This generic 403 error means that the authenticated user is not authorized to use the requested resource. A substatus code in the IIS log files should indicate the reason for the 403 error. If a substatus code does not exist, use the steps above to gather more information about the source of the error.

Both applications are running as WebApp in Azure and there are separate Kentico 10 instances. One very noticeable issue is the request url "my-kentico-app__f394" this is going from who know's where from as nowhere in our system is such url declared "my-kentico-app" is the only valid and working url.

This looks internal Kentico issue to me, does anyone experiencing same errors in the Application Insights as well?

Thank you Ondrej

   Please, sign in to be able to submit a new answer.