Can anyone tell me how best to optimise my CMS site for web spiders e.g Google?

I have some software running currently that reports on the ISS logs produced, and it seems that most search engines are looking for presense of robots.txt. Should I use this file with Kentico and if so what should I Dissallow?

Thanks