# Allow all robots to search everything, we do not disallow anything. User-agent: * Disallow: # Allow robot visits between 2:00 AM UTC and 10:00 AM UTC. # Which is 7:00 PM PST to 3:00 AM PST Visit-time: 0200-1000 # No delay limits so far for robots. # Crawl-delay: 5 # Request-rate: 1/5