johnyboy — 2013-07-04T06:05:44-04:00 — #1
Is there any reason why curl wouldn't be able to handle this URL? (I've removed the domain name -- not sure why, just have):
In my browser it works. Using curl, in a way that works fine for other URLS, it goes semi wrong. It's impossible to say exactly how it goes wrong because my local library server (the site I'm accessing) does stuff with it behind the scenes, or doesn't in the curl case. Basically, it does the right thing when I access that URL in my browser but not the right thing via curl.I know there's not a lot to go on there but maybe curl messes with URLs in some way?(separately, this form, for submitting posts, doesn't recognise new line/returns -- they get lost :/ Safari 6.0.5 on Mac)
johnyboy — 2013-07-04T06:17:50-04:00 — #2
Just to add, that URL, that's how it is in the webpage in the href attribute of an anchor element -- verbatim. I've tried messing round with it in various encoding/decoding ways. No joy. Any ideas? It's a GET type URL obviously. It's obviously the query bit, after the ?, where something is going wrong. Do browsers do something with URLs like that when they're clicked on which isn't being done for curl?
johnyboy — 2013-07-05T09:54:22-04:00 — #3
Aha, cracked it at last. Maybe predictably, it wasn't the URL getting mangled it was the cookies failing to get through because of http and https and the fact that the cookies were set to FALSE for the secure field which presumably stopped them getting through for the https requests.
Haven't actually sussed out how to set curl cookies to secure TRUE (the fourth tabbed column in the text cookie file) yet -- I did it manually (find and replace in text editor) to see if that was the problem
johnyboy — 2013-07-05T10:34:12-04:00 — #4
That turned out to be rubbish. Nothing to do with secure column. Anyway, it is fixed now. Thanks.
johnyboy — 2013-07-05T10:59:13-04:00 — #5
Finally, finally properly sussed it. The url got from the webpage had & 's in. They needed converting to & 's before using with curl. Simple. Silly me.