This session at SES NY went over how search engines detect and deal with duplicate content and the strategies you can use to reduce duplicate content on your site.

If your server is not properly configured to give a proper response for a robots.txt file your site will not be indexed by Google.

There are a number of ways search engine spiders can become confused crawling your site. This post goes over common pitfalls and some best practices so spiders get a clear understanding of your site.