So, full disclosure: I probably set JeffreyBarke.net up wrong. Or, at least not in a “typical” way. However, it’s been the way it is for a long time now and I never thought there was a problem until yesterday, when I was checking Google Webmaster Tools for crawl errors. It turns out I was getting two 403 Forbidden error codes on two pages: Code and Tools. At first, I didn’t believe the results because the pages render fine, but I checked the actual headers and, yerp, there was the 403.
Before I talk about how I got the pages to return proper 200 response codes, I want to give a little background on how JeffreyBarke.net is structured and how it got the way it is. The site is primarily powered by WordPress, which is installed in the web root. However, there are a number of directories (some legacy, some not) that are also in the web root. I have WordPress set up to use “clean URL” permalinks (achieved via mod_rewrite). Sometimes, the WordPress mod_rewrite URLs overlap with existing directories, and this is causing the 403 response codes.
JeffreyBarke.net got this way because I used to have WordPress installed in a /blog/ subdirectory and the site root was custom-powered by a mix of flat and dynamic files. I got tired of this arrangement and decided to run everything through WordPress by doing a clean install at the web root. However, I also wanted certain demos, tutorials, code and tools which couldn’t be integrated with WordPress to remain at their current URLs.
This means I have a physical directory that looks like the following excerpt:
/
index.php
tools
codeigniter-encryption-key-generator
index.php
wp-admin
wp-content
wp-includesSince there’s no index.php file in the tools directory, when you navigate to http://jeffreybarke.net/tools/, mod_rewrite routes you to the WordPress page with the tools slug. When you navigate to http://jeffreybarke.net/tools/codeigniter-encryption-key-generator/, the actual index.php file is returned.
Here’s the interesting part, though: since I disabled directory browsing in the root .htaccess file and there’s no index.php, a request for /tools/ should return a 403 Forbidden response code. I’m kind of surprised that the page renders a’tall.. If I enable directory browsing, the page doesn’t render and I get the standard Apache directory view.
Ok, enough background—how do I have my cake and eat it too?
These are the mod_rewrite rules WordPress automatically generates for my permalink structure:
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPressThe second RewriteCond checks that the URL isn’t referring to an existing directory, which, in the case of both /code/ and /tools/, it is. It’s this second RewriteCond I want to disable, but only for specific directories.
This can be achieved by adding the following lines to the .htaccess file that contains the WordPress permalink structure:
# Fix 403 errors on existing directories; WordPress overrides.
RewriteCond %{REQUEST_URI} ^/(code|tools)/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule . /index.php [L]
.