Complex marketplace site idea - help determine best path for long term effectiveness

Hey guys I’m new here and fairly inexperienced with website design and programming. Basically I have an idea for a site and no money to hire someone, but lots of time. I’m not sure if a marketplace is the right description either! I’m not going to get into the products at this point since they’re not relevant just yet. This idea stemmed from all the headaches I’ve had trying to purchase refills or replacements for these products. Here’s the story; I purchase product X from company A on a regular basis. Then I find out when I go to make my next purchase that they are out of stock. Now I have to decide how much time I want to invest into finding a new place to order from. This involves researching who is selling product X, their prices, availability, customer service, shipping etc. Even after that I still have to deal with the possibility that their quality, product, and/or services are not equivalent to product X from company A. This also stems from my rage at the internet for being the biggest haystack of needles!

So here is what I would like to do; make a site that is geared towards the consumer by consolidating as much information into one place as possible, make the information relative to each other, provide a place where consumers can leave reviews without fear of having their review removed because it reflects poorly on a product or company, and hopefully create a system for making product recomendations similar to how movies can be recommended by Blockbuster or Netflix. That last part would be really cool if it could compare users preferences and make recommendations based on users that are similar in their choices.

I would rant about my personal views and back story to all of this more but it doesn’t help get this project done so I’ll try to keep it minimal.

Considering I’m gearing this towards consumers, I’m sure that at least a few companies selling these products will not be happy with me when a user chooses to purchase elsewhere because they find that company B sells it for less and has a better customer service rating or any other reason that someone might jump ship.

The other goal of this as I mentioned briefly is the situation where the company is out of stock and you want to quickly find a good replacement even if you plan to continue using company A once they get more stock.

So far I have started but I’m not sure what the best way to do this is or where I should be starting. Since I’m going to assume that at some point there will be some company that isn’t happy with their reputation on the site, I want to use a method to obtain the data without having to access their database. In a perfect world each of these companies could allow access to their databases to import the data into my database but even if that were the case, I’m sure there are many different formats that would have to be dealt with. So far I have used OutWit Hub to deal with the html source code and export what I need into Excel. I have also been messing with Excel directly to query the sites but so far none of them are nice for Excel to import.

This is the first big obstacle for me is the data collection. I need something that doesn’t require special access or permission and reduces as much repetitive manual labor as possible. It would also be a plus if the set up labor can be minimized but my main concern is developing something that doesn’t require tons of work to make changes to. Right now with either option I have used there will be lots of set up labor and then lots of labor to verify the data or find problems. In Excel I can create web queries for each page on each site and then visually look for the data I need and link those results to a different sheet which would be saved. This is slower than OutWit Hub for the moment but I think it might be faster in the long run. OutWit Hup Pro has macros which I think would be even better but I don’t have money to purchase it. Either way both would require almost starting from scratch for a particular page or rule if the content I’m looking for changes or is moved and I would like my site to be able to update at least daily with all of these sites to verify the data is current and update if it’s not.

The next thing is how to store the data, then get it onto my site.

Then things move on to site design and how to display the data with sorting and search functions.

The initial goal is to get the data and get it on my site where it is usable. Once traffic starts showing up I would be more concerned about user reviews and after that the recommendations thing. The important part in all of this is developing this project with these additions in mind so I’m not having to go back and change a bunch of stuff when I go to add a new feature.

If anyone is wondering, yes I do want to make some money off of this. My thoughts are to set up affiliate deals with as many sites as possible. I don’t know exactly how that works either but I will not build this site to promote ANY PARTICULAR product. My goal is not to get in bed with some retailer but rather what I would like to see is my affiliate links on their specific product pages next to a direct link to that products page for that company. The idea is to let people know that I am not pushing any products but if they would like to help out, they can purchase through the affiliate link so I can receive some money. This way they still find what THEY want and THEY choose if they want to contribute to me by using my affiliate link. Personally I could care less if someone purchases from one company instead of another but I do appreciate when people show thanks when they feel I’ve helped them out. Hopefully some of the affiliate links I would be provided could offer a lower total cost to the user which I’m assuming would really give a user the incentive to contribute but it’s just a hopeful thought.

Well thanks for reading and any input you guys can offer. Let me know if I should elaborate more on my goals etc.

Hi MeltManBob,

The first thing I would recommend is doing more market research and to read http://gettingreal.37signals.com/toc.php

What makes your app different to the competitors?
Let’s assume your stocking office products like pens and paper, what are you going to do better than office suppliers?
Manufacturers would pay to advertise and provide supplies through your site if you can sell their products.
I think one of the best ways to make this successful would be to allow orders / deliveries online.
I doubt that you would gain traction if it’s just a price comparison site.

After you know more about the key features of the site then you can make a start on the design.

Welcome to Sitepoint, and good luck :slight_smile:

  • As far as I know there is no competing apps.
  • I don’t think it’s about what I’ll be doing better than the suppliers since I’m wanting to do something that they don’t. None of them want to show their products side by side with their competitors because then it would be too easy for the consumer to see who is ripping them off either with their price, reviews, etc.
  • Down the road I hope to have most of these suppliers willing to create affiliate links that will offer a discount to the consumer so they have a financial incentive to purchase through me and not just a slight moral obligation for the help they took advantage of. I’m not opposed to taking orders and passing that on to the particular supplier instead of affiliate links. My attitude in that respect is what ever is easiest and consistent for the end user. IE I don’t want half with affiliate links and half where you actually order on my site. If that’s how it has to be then so be it but ideally I would like one style of whatever you want to call it.
  • I disagree, this market is one that is very convoluted and I know a lot of people who would find something like this useful. To be honest I’ve been wanting to do this for myself simply because of how often I have to go through the process of tracking down what I need/want. I just figured if I’m going to waste that much time, may as well try and do it so I can help others and see if I can generate some income from it.

You mention that when I know more about the “key features” then I could start on a design, do you mean the actual visual representation of the site? If you are referring to the actual functionality of the site then I’ll try to list what was in my post more simply:
1)Database of products and companies that can be searched and sorted with multiple levels of search criteria. ex: search - “specific item name”, sort by price then by company customer service rating then warranty period then product rating etc. Must be efficiently organized and intuitive to use.
2)Database of user reviews on products and companies.
3)Relevant ratings. ex: you’re looking at purchasing a new product, both have good ratings but good ratings from another user who shares similar ratings on previous products common to both users would hold more weight. ex: Person A and person B statistically rate 75% the same, person A and person C statistically rate 10% the same, chances are person A will be more satisfied with person B’s recommendation as compared to person C’s. I hope that makes sense :slight_smile:

Here’s a hypothetical scenario: Lets say I want to create a site where I catalog as many burger joints as I can for the same general purpose as I’ve stated. Initially I would have just basic information on the more common places like McDonalds and Burger King. I would make a list of all of their burgers, names, prices and then make that data available on the site in a sortable format. I would continue to add more places and burgers and I find out about them while also trying to increase the amount of data per burger and company in the database. Eventually it would have the ability to list all of the toppings available for each burger, list of parts (1 meat patty, 2 slices of American cheese etc.) Lets go one step further and say that eventually I’m able to provide nutritional information for each ingredient also. Outside of relating to the product there would eventually be a rating system for the company, particular location, workers, facilities etc. The rating system would be both an actual rating along with written reviews.

Now, the main page would be very simple in it’s functionality; Find burger, Find burger joint, Find burger company. Depending on what you pick will determine the information that is used and once it is displayed, the user can sort through it however they like. If they want to find the highest user rated burger, no problem, find which local joints to avoid because of crappy customer service, crappy food prep, crappy ingredients etc.

It wouldn’t surprise me to know that there are similar sites for various markets but I have yet to this day come across a site that was the one stop shop for a category of products. One thing I want to do better than places like Amazon is to not duplicate the same items because they were listed differently. As far as I know even Amazon doesn’t have the kind of functionality I’m talking about, not even close. I’m not trying to just make shopping easier, I’m trying to eliminate as much of the tedium as possible and make it so the user only has to invest enough time for them to make the judgement calls that only they can make. Personally I rarely even get to that point because the amount of work that is required to become well informed; usually I do enough to find something that will meet most of my important needs and that’s it. What I’m getting at is eliminating the situation where you start compromising on what you’re looking for simply because the amount of time it takes to find. Some people don’t mind waiting long amounts of time researching for the best deal but most of us do mind. I mean how cool would it be if you could pull up a site that gave you the ability to search for a car you wanted, find just about every one in your city, know who was selling it, how long the warranty is, what the user rating of that warranty has been, the customer service rating of the dealership, service department etc. all on one site and quickly sort through to find EVERYTHING that meets ALL of your criteria so you can spend your time making the important choices. I hope this clears up what my goals are.

I have a pretty good idea of what point A and point B are. I need help figuring out which path will not shoot me in the foot down the road as I try to incorporate more information and features since it would be ridiculous for me to roll out my point B from the get go. I’m saying that because I don’t have the money to hire a pro and have it done ASAP and with everything I want and because I will be the one doing most of the work to get this going. I really hope that if this takes off a bit that eventually I can hire a pro to streamline and clean things up or improve things but for now I need to work on functionality.

Step 1 - Determine best method for collecting data - scraping or some other method (this is an area I am unfamiliar with and need suggestions)
Step 2 - Determine best method for storing data - offline database and online (I’m familiar with Excel for it’s basics and I think it will be ok for offline storage. For now I’m using MYSQL database online but this means figuring out how to import data from excel which I’ve had some success figuring out.)
Step 3 - Determine best method for updating existing data collection
Step 4 - Determine best method for retrieving data from online database
Step 5 - Determine best method for displaying data
Step 6 - Determine best method for adding reviews and ratings
Step 7 - Determine best method for income (affiliate links vs direct ordering through my site)
Step 8 - Determine best method for relating reviews and ratings to each other.

I’m sure that data collection will be much easier if the site owner in question is cooperative and allows more direct access. I would also like to find some way of doing verified purchase reviews such as Newegg. I want to avoid reviews and ratings that are irrelevant because some pissed off customer wants to leave nasty reviews for things they never even purchased, just to make the company look worse. I know I will never get rid of that completely but I’m hoping with relating the reviews to each other that will help. Granted that would only work for people who create an account so their info can be saved but at least for those who really make use of the site, they can sign in and see recommendations based on like minded people instead of the haystack of anonymous reviews that may or may not be legit.

“- As far as I know there is no competing apps.”

Should read - As far as I know there are no competing apps.

Edit - It slapped me in the face after the fact :frowning: There are probably more…

What other information should I provide in order for people to make recommendations as to what design path to take? Any path is daunting for me so I was hoping that someone who knows this field and the tools could explain what design paths offer what benefits. I just don’t want to start down a path that ultimately leads me to a dead end where I have to go back and re-learn a totally different path that I could have avoided if someone had just set me straight to begin with.