User-agent: * 
Disallow: /
User-agent: W3C-checklink
Allow: /