Forums (Obsolete)


Member	m00nstar - 9/24/2008 7:44:35 PM Robots.txt Hello! Just a quick one. How/where would I create a robots.txt file in Kentico? I can easily add a file to the server itself if not supported within the CMS, but where? Cheers Daleth.


Kentico Support	kentico_jurajo - 9/25/2008 3:25:25 AM RE:Robots.txt Hello, You need to place the file to the root of the web project. Moreover, this file have to be created manually - there is no option how to create it with Kentico CMS right now. Best Regards, Juraj Ondrus


Member	m00nstar-hotmail - 9/25/2008 3:57:46 PM RE:Robots.txt Thank you. I'm wondering where the project root is. This piece of documentation says it should be here: \Inetpub\wwwroot\<web project> But I can't find the folder I need there. In a standard install, is there anyother place it could be? Does it matter what version you are using? Thanks again!


Kentico Support	kentico_jurajo - 9/26/2008 3:02:42 AM RE:Robots.txt Hi, Which folder you are not able to find? The example is just the default setting. If you have installed your web into the root there is no <web project>. The meaning of the brackets is that it is optional and the folder name could be custom. You only need to place the file to the root of your web project. Best Regards, Juraj Ondrus


Member	m00nstar-hotmail - 9/29/2008 4:27:47 PM RE:Robots.txt Thanks!


Member	mcutter-global-id-group - 12/16/2008 11:58:30 AM RE:Robots.txt When I upload a file robots.txt to the root level then it gets served by the url ~/robots.aspx, and not robots.txt. I can easily put in a url rewrite from robots.txt to robots.url but am wondering if there is a way to do this within Kentico itself.


Kentico Support	kentico_radekm - 12/22/2008 1:45:45 AM RE:Robots.txt Hello, there is no native way how to do it within Kentico itself in present time. We plan to add this feature into some of future releases of Kentico CMS. Best Regards, Radek Macalik

Member

Vispi - 4/24/2010 5:25:21 PM

RE:Robots.txt

Hi guys

Just wanted clarification on the robots.txt

Would a robots.txt file with the following content
User-agent: *
Allow: /default.aspx
Disallow: /

Will this file serve the website and all the pages created within the CMS, while blocking the remaining folders such as CMSPages, etc?

Also, my blog entries on the website do not appear to be getting indexed by search engines. How can I enable them to be generated on the Sitemap or allow them to be crawled by search engines?

Many thanks,
Vispi


Kentico Support	kentico_radekm - 5/25/2010 7:02:56 AM RE:Robots.txt Hello. It should be like this for your purpose: User-agent: * Disallow: /CMSPages/ As for how to show cms.blog document in Site map web part, you only need to set web part´s property "Document types" like this: CMS.MenuItem;CMS.Blog; Best Regards, Radek Macalik

Certified Developer 8

manoj-pulldigital - 1/18/2011 11:29:11 AM

RE:Robots.txt

Hi,

Another really esay solution is to add a robot.txt as a file type in the cms desk. To do this you need to allow file type under the root document type in site manager> documenttypes search for 'root'

Upload your robot.txt file

You than need to allow the url extention property of .txt in the url property of the uploaded robot.txt file.

This is the only solution that I could think of if you have multiple sites on a single instance of kentico... interested in hearing any other thoughts...

Certified Developer 11

jason-datausa - 8/1/2011 4:16:12 PM

RE:Robots.txt

Hello,

You can also use the rewrite module if you're hosting your website on IIS 7 to create a different robots.txt file for each website. It's very easy to set up! If you're unfamiliar with the rewrite module, I've written a more detailed post on how to accomplish this here: http://www.kenticosolutions.com/Developer-Tips/Tip/July-2011/Multiple-Websites-and-Robots-txt-no-problem.aspx.


Kentico Support	kentico_zdenekc - 8/4/2011 8:51:38 AM RE:Robots.txt Hi Jason, That's nice, thank you for sharing your solution with us here. www.kenticosolutions.com is definitely great information source for all Kentico CMS developers and users. Regards, Zdenek Cetkovsky