#deny all spider user-agent: * disallow: /