How to Disallow All Except for robots.txt in Your Site Crawling Rules? or Is There a Way to Disallow

18 Replies, 1899 Views

That’s the right way to disallow all except for robots.txt, but it’s more of a suggestion than a hard block.

If you’re serious about blocking everything else, consider server-side restrictions. NGINX or Apache rules can enforce it strictly.

Messages In This Thread



Users browsing this thread: 1 Guest(s)