rainy_hu — 2012-10-29T13:08:01-04:00 — #1
Dmoz export file is too big for PHP I think.
It is over 4GB and my PHP cannot handle such big XML file.
How can I import or convert it to mysql?
Or how can I using php 5.3 dump the odp data to mysql 5.0
kylewolfe — 2012-10-29T14:13:23-04:00 — #2
Is the file local to the mysql database you will be importing to? Exactly what format is the file in? I'm thinking you should be able to do a command line import right from mysql. I've done this with CSVs of several gigs before.
rainy_hu — 2012-10-29T21:19:11-04:00 — #3
The ODP(DMOZ) data file format is RDF. Url is http://rdf.dmoz.org/rdf/
How can I using php 5.3 dump the odp data( file format is RDF) to mysql 5.0
cpradio — 2012-10-30T07:27:37-04:00 — #4
Checkout the following results I got from a google search
cups — 2012-10-30T07:34:55-04:00 — #5
You will likely need to get your head around SPARQL in order to break up the RDF and iterate through the file.
This search RDF to SQL PHP turns up some solutions, including ARC2 which is designed for that type of work. Another search: PHP ARC2
Setting it up might be a bit of a pain.
IIRC there were online tools around to do this a few years ago.
If the RDF is coming from a datastore with a SPARQL endpoint (which they usually do) then you might be able to grab the data over an API, and do it in chunks too.
kylewolfe — 2012-10-30T09:41:49-04:00 — #6