bluelude1 — 2011-07-12T20:08:17-04:00 — #1
Since Amazon instituted their new throttling system about 9 months ago for their API that limits calls to 2000 an hour + 500/ $1 sold - Capped at 25,000 calls per hour. What are the available alternatives? I noticed that you can request a daily data feed to be downloaded but that could require substantial daily processing. Is there any other way to go about accessing information for a price comparison feature outside of scraping, which doesn't make sense in itself?
My concern is I utilize Amazon's API and I actually have success and now I have an issue maxing out my allotted hourly calls. I have read several instances where people have access to the content for the first 15min of every hour before their traffic maxes out their calls.
Thanks in advance
ted_s — 2011-07-12T20:11:45-04:00 — #2
I'm in the process of pulling together product comparisons myself so this resonates well.
While it may take additional work / effort it seems like the daily feed would be the ideal solution to get everything you need and speed up the loading without relying on any updates. Of course it comes at the risk of real time changes but it's guaranteed not to fail.
Alternatively if you bring enough revenue I'd just drop Amazon a line and ask for more calls. Limitations were placed because, as they put it, a very small number of sites were eating up a very large number of inquiries with virtually no revenue. If you bring value they'll return the favor.
bluelude1 — 2011-07-12T20:40:21-04:00 — #3
Do you think it makes sense to start with the API and try to implement the use of data feeds before traffic approaches the 25k threshold ...if they aren't interested in upping my number of calls?
If I went the data feed route...what are we talking about difficultly wise to develop a system to process the feeds?
From what I have heard the API is the lightest of the two options to implement as the product information never has to be downloaded, processed & sit on your system. It just operates as more of a plug and play system that calls for information as needed. Please correct me if my understanding is incorrect.
BTW - Thanks for such a quick response
ted_s — 2011-07-13T01:23:17-04:00 — #4
If you are only showing one of a store parsing a data feed will be a pain but it's a one time effort, much like writign an API routine. In a sense you are making a script to read and record the data and then your own API to call it back up. Using their API directly is indeed simple [for coding that is] but has the drawback of having to be run on every store and load.
If you compare many stores it becomes more complex as you're now cross storing them all but is likely more powerful as you can now compare and rank all the options against each other.
Of course 2,000 calls [at the minimum] is no small number. If even 0.5% convert at a $50 order you'll get to the 25k max which would be $62,500 an hour in sales.
The trick is to call the API when needed, not every time someone loads any page on your site.
swishguy — 2013-04-06T18:09:33-04:00 — #5
ListBeat offers a data download of Amazon products - you could try building your own database and avoid the API altogether.