Path appended to a path - probable mod_rewrite

Server Config
1

I have a strange set of links and do not know what is causing them or how to find out what is causing them !

Xenu says:

http://www.rubblewebs.co.uk/imagemagick/snippets/www.rubblewebs.co.uk/imagemagick/snippets/index.php
error code: 404 (not found), linked from page(s):
http://www.rubblewebs.co.uk/imagemagick/snippets/www.rubblewebs.co.uk/imagemagick/snippets/Resize_snippets_f_52.html

If I click on the second link ( which is still wrong ) a page loads with the URL but if I click the first link Xenu is correct the page does not exist.

All the incorrect URLs are of the format:
http://www.rubblewebs.co.uk/imagemagick/snippets/www.rubblewebs.co.uk/imagemagick/snippets/Resize_snippets_f_52.html

I have a rewrite rule on the folder but that works for all the other files with a similar rule just the resize one fails:


Options +FollowSymlinks
RewriteEngine On
RewriteRule GD_snippets_f_(.*)\\.html$ GD_snippets.php?f=$1
RewriteRule Creation_snippets_f_(.*)\\.html$ Creation_snippets.php?f=$1
RewriteRule Modify_snippets_f_(.*)\\.html$ Modify_snippets.php?f=$1
RewriteRule Imagick_snippets_f_(.*)\\.html$ Imagick_snippets.php?f=$1
RewriteRule Text_snippets_f_(.*)\\.html$ Text_snippets.php?f=$1
RewriteRule Resize_snippets_f_(.*)\\.html$ Resize_snippets.php?f=$1
RewriteRule php_snippets_f_(.*)\\.html$ php_snippets.php?f=$1
RewriteRule Optimize_snippets_f_(.*)\\.html$ Optimize_snippets.php?f=$1
RewriteRule Watermark_snippets_f_(.*)\\.html$ Watermark_snippets.php?f=$1
RewriteRule Utilities_snippets_f_(.*)\\.html$ Utilities_snippets.php?f=$1
RewriteRule psd_snippets_f_(.*)\\.html$ psd_snippets.php?f=$1
RewriteRule Mask_snippets_f_(.*)\\.html$ Mask_snippets.php?f=$1

IndexIgnore *

I also have a correct URL in the list: http://www.rubblewebs.co.uk/imagemagick/snippets/Resize_snippets_f_6.html as well as the bad ones.

If I comment out all except: RewriteRule Resize_snippets_f_(.)\.html$ Resize_snippets.php?f=$1
I do not get any errors.
If I leave everything and just comment out RewriteRule Resize_snippets_f_(.)\.html$ Resize_snippets.php?f=$1 all is OK again.

The problem only seems to be when the resize rule is in with the other rules.

Anyone have any idea how to track this problem down?

2

Hi Anthony

This is a bit of a stab in the dark but try setting a base href for your links.
add the following into your head tag:


<base href="http://www.rubblewebs.co.uk/">
3

Adding base gave me loads of 404 errors - it seems to have removed the folder so http://www.rubblewebs.co.uk/imagemagick/notes/display.php becomes http://www.rubblewebs.co.uk/notes/display.php

I ment to say if I moved the resize rule to the top of the rewrite list it I had a load of errors; going back to do it again to record the errors they had gone and I had the double links back!

I am also using linkchecker 7.5 now as I can view the results easier than with Xenu.

4

Rubble,

RewriteBase is designed to UNDO a mod_alias redirection for mod_rewrite to work with the result. It’s more of a PITA than anything useful (IMHO).

Are you aware that, without any Last flags, you’re ANDing all your RewriteRules? Okay, maybe not (there IS an automatic AND for RewriteCond statements and their associated RewriteRule) but play it safe!

Each one of your RewriteRules have regex f_(.)\.html. Is this meant to accept f_.html, too? If not, PLEASE try NOT to use the “catch all” (.) as it’s more trouble than it’s worth! Now, that said, you’ve done well by placing this in this code in the .htaccess in the DocumentRoot which would cause all your redirections to be relative to the DocumentRoot, too!

To the problem (www.example.com repeated within the path and no excuse for such nonsense in the mod_rewrite code), I’ve only seen that in Apache when it needs a restart. If you control the server (i.e., VPS or better), give it a quick restart, otherwise, ask your host to do that for you.

Regards,

DK

5

Thanks for the info DK and here are the results of my tests:
Restarting Apache had no effect
Adding [L] had no effect - but as you say there should be an [L] automaticaly inserted anyway
RewriteRule GD_snippets_f_(.php)\.html$ GD_snippets.php?f=$1 [L] ( all rules changed but just posted the first line) Sorted the problem BUT broke all the other snippet links with a 404 error!

Now, that said, you’ve done well by placing this in this code in the .htaccess in the DocumentRoot which would cause all your redirections to be relative to the DocumentRoot, too!

This .htaccess code is within the snippets folder rubblewebs.co.uk/imagemagick/snippets

6

Rubble,

Restarting should have worked.

[L] is NOT automatic, AND between RewriteConds and RewriteRules are!

Once more in English, if you please. Does that mean that you only added the Last flag to the first RewriteRule and NOT to each of the other RewriteRules?

Test URIs? We need that here to try to follow what’s happening during your testing.

Finally, the .htaccess MUST be in the path to the requested file, not off in Neverneverland! From your description, the URIs you’re attempting to redirect are in your DocumentRoot so your .htaccess must be located there to have any effect at all.

Regards,

DK

This .htaccess code is within the snippets folder rubblewebs.co.uk/imagemagick/snippets[/QUOTE]

7

1/ I had put the [L] after every rule ( this is the current .htaccess file on the server ):


Options +FollowSymlinks

RewriteEngine On
RewriteRule GD_snippets_f_(.*)\\.html$ GD_snippets.php?f=$1 [L]
RewriteRule Creation_snippets_f_(.*)\\.html$ Creation_snippets.php?f=$1 [L]
RewriteRule Modify_snippets_f_(.*)\\.html$ Modify_snippets.php?f=$1 [L]
RewriteRule Imagick_snippets_f_(.*)\\.html$ Imagick_snippets.php?f=$1 [L]
RewriteRule Text_snippets_f_(.*)\\.html$ Text_snippets.php?f=$1 [L]
RewriteRule Resize_snippets_f_(.*)\\.html$ Resize_snippets.php?f=$1 [L]
RewriteRule php_snippets_f_(.*)\\.html$ php_snippets.php?f=$1 [L]
RewriteRule Optimize_snippets_f_(.*)\\.html$ Optimize_snippets.php?f=$1 [L]
RewriteRule Watermark_snippets_f_(.*)\\.html$ Watermark_snippets.php?f=$1 [L]
RewriteRule Utilities_snippets_f_(.*)\\.html$ Utilities_snippets.php?f=$1 [L]
RewriteRule psd_snippets_f_(.*)\\.html$ psd_snippets.php?f=$1 [L]
RewriteRule Mask_snippets_f_(.*)\\.html$ Mask_snippets.php?f=$1 [L]

IndexIgnore *

2/ There are some test URL’s in the first post:

Incorrect working URL with the address appended generated from the rewrite.
http://www.rubblewebs.co.uk/imagemagick/snippets/www.rubblewebs.co.uk/imagemagick/snippets/Resize_snippets_f_52.html

A correct working URL generated from the rewrite.
http://www.rubblewebs.co.uk/imagemagick/snippets/Resize_snippets_f_52.html

3/ The .htaccess file is within the folder containing the snippets:
Root/imagemagick/snippets/.htaccess
Root/imagemagick/snippets/Resize_snippets.php

4/ The links can only be found using a link checker and so I could leave it as it is but it would be nice to tidy things up.

Thanks

I must reiterate that the Resize_snippets.php is the only problem; all the other rewrites work as they should.

8

Rubble,

Okay on the new .htaccess code. Again, though, the Options should already be enabled in the server’s conf file. As a matter of technique, I also put all core directives (like your IndexIgnore) before the mod_rewrite code as it has precedence.

Also, unless you’re posting PHP code, you should be using [noparse]

 ...

[/noparse] wrappers.

Your last sentence is the key: The string in red is repeated and includes the domain name (www.rubblewebs.co.uk/imagemagick/snippets). However, with the code you have, even that should have been matched and removed as is happening with all your other code.

As a test, would you please “replace a string with another string” before you start your mod_rewrite code sequence:

RewriteRule ^(.*)/www\\.rubblewebs\\.co\\.uk/(.*)$ $1$2 [R=301,L]

Like the other code dealing with this errant URI, this code is attempting to remove specifically your www’d domain from the {REQUEST_URI} string. The R=301 in the flag is to allow you to see whether the domain’s been removed or not. If you find that some have omitted the www. from your domain, try

RewriteRule ^(.*)/(www\\.)?rubblewebs\\.co\\.uk/(.*)$ $1$3 [R=301,L]

Regards,

DK

9

I think I am going to admit defeat DK :frowning:

I tried:

Both versions of your extra code to remove the extra URL and it had no effect

I tried moving the .htaccess to the root folder - no change

Rewrote the .htaccess file incase there was a strange character in it somewhere that was not showing up (I did check with view all characters first but thought it was worth a go) - no change

I have restarted Apache a couple of times during the tests - no change

Moved the Resize to the last on the list - no change

Moved the Resize to the first on the list ( after your extra code posted above ) - no change

Removed the Resize rule - error on finding the resize pages only

Removed everyting but the resize rule - error on all the other snippet links but not the resize one.

Current file contents:


RewriteEngine On

RewriteRule ^(.*)/www\\.rubblewebs\\.co\\.uk/(.*)$ $1$2 [R=301,L]
Rewriterule GD_snippets_f_(.*)\\.html$ GD_snippets.php?f=$1 [L]
Rewriterule Creation_snippets_f_(.*)\\.html$ Creation_snippets.php?f=$1 [L]
Rewriterule Modify_snippets_f_(.*)\\.html$ Modify_snippets.php?f=$1 [L]
Rewriterule Imagick_snippets_f_(.*)\\.html$ Imagick_snippets.php?f=$1 [L]
Rewriterule Text_snippets_f_(.*)\\.html$ Text_snippets.php?f=$1 [L]
Rewriterule php_snippets_f_(.*)\\.html$ php_snippets.php?f=$1 [L]
Rewriterule Optimize_snippets_f_(.*)\\.html$ Optimize_snippets.php?f=$1 [L]
Rewriterule Watermark_snippets_f_(.*)\\.html$ Watermark_snippets.php?f=$1 [L]
Rewriterule Utilities_snippets_f_(.*)\\.html$ Utilities_snippets.php?f=$1 [L]
Rewriterule psd_snippets_f_(.*)\\.html$ psd_snippets.php?f=$1 [L]
Rewriterule Mask_snippets_f_(.*)\\.html$ Mask_snippets.php?f=$1 [L]
Rewriterule Resize_snippets_f_(.*)\\.html$ Resize_snippets.php?f=$1 [L]

I am starting to confuse myself now and as I say the problem only shows up when using a link checker.

10

Rubble,

Sorry, we tried. There must be something in the mod_rewrite regex parsing code which chokes on the domain being in the path. Very strange behavior (not being able to remove it with the above mod_rewrite code) but, as you say, it’s just the odd ID-ten-T request so there’s no real worry.

Regards,

DK

11

Thanks a lot for the help anyway DK.