Yep, that’s the classic method for disallow all except robots.txt. But like everyone’s saying, it’s not 100%.
For tighter control, use .htaccess or server configs to physically block access to other files.
Also, check out Bot Protection services—they’re worth it if you’re getting hammered.
For tighter control, use .htaccess or server configs to physically block access to other files.
Also, check out Bot Protection services—they’re worth it if you’re getting hammered.
