Drupal or Custom CMS for Niche Social Network?

I think it’s very good to plan these things out. My major programming projects (i.e., projects I can’t just rewrite from scratch in six month) I generally plan out a minimum of 3 years in advance so I can plan for things down the line (as best I can).

I disagree with brense, completely on one point, and half way on another. =p

The point I disagree on is Apache. Apache isn’t that great at serving tons of files. nginx is much better (especially at static files). We did lots of benchmarking the past couple months and say a substantial difference (around a minimum of 10x depending what we were testing).

The part I half agree is PHP and MySQL. Both of these are more than capable of handling any amount of traffic (at least as well as anything else). However, there are limits of everything, which is why you need to plan out a clustering strategy, even if it’s a while until you release it. Nothing worse than realize you are two months away from needing to cluster and you realize it’ll take six months to get the code to be able to handle that. =p Generally it isn’t PHP or MySQLs fault as it is something else (CPU limits, RAM limits, I/O read/write limits, etc.) Facebook isn’t run from one server for a reason. :wink:

According to the below:

http://stackoverflow.com/questions/3479720/what-database-does-facebook-use

Facebook uses mysql along with other technologies. I think they are up to 30,000 servers at this point.

Here is a great article that breaks down what facebook has done to scale. I just found this one and it has more detail than the other link:

http://royal.pingdom.com/2010/06/18/the-software-behind-facebook/

Yeah, when you get as big as Facebook you start a whole new layer of crazy. Basically you have to start building your own customized systems from the ground up at an even lower level than PHP. But you can put that off at least until you are approaching 10 million users. :wink:

Samanime, that gave me a good laugh. I have no designs on getting anywhere near facebook. I am just trying to build something that is very functional and useful for a targeted niche audience. There is only one facebook and that ship has already sailed. If I build the tool and people find it useful, then there is satisfaction in that.

You can try cubesql http://www.sqlabs.com/cubesql.php
It’s designed for high performance and since it’s sqlite i think it should be easy to import/export to/from mysql if you decide to use it later or if you want to switch back.

Drupal can scale just as any other CMS, or software system.

Do you really think your going to build a system more capable than Drupal - which has hundreds of contributing members analysing the design? Not saying it’s not possible - of course it is. However if you have to ask this question you are obviously not intimately familiar with Drupal’s design nor CMS in general - I would assume.

You can scale WordPress too just in slightly different ways than Drupal - every CMS makes a compromise somewhere. So Unlesss you have deep experience in implementing large scale apps in CMS and know exactly where the pitfalls are and how to improve on them - building custom is simply re-inventing the wheel, IMO.

Drupal will need to be studied to understand it’s architecture which is somewhat unique. It has a learning curve but so will building from scratch as you encounter scaling issues, maintainable issues and the like.

The choice is ultimately a personal one, but professionally speaking, if your building at the expense of someone else - Drupal for the win. Personal project go custom for the experience

Cheers,
Alex

PCSpectra (Alex), you assumed incorrectly. I am very familiar with Drupal and have used it extensively for the last 5 years, that is why I am considering using something else. Drupal is not known for scaling well when you have a lot of authenticated users. Do some research and try to find a high volume social networking website that is based on Drupal. I have been looking for examples for weeks. That is why I posted the question in the first place.

The major one social networking example listed over on Drupal.org is teamsugar.com. They did a case study and they had to hack Drupal to death because of performance and scalability issues. Teamsugar states that in some areas, that their site does not even act like Drupal anymore after their changes. They also say that their non social networking sites are fine (they have magazine type websites also that use Drupal), but they had to do extensive customization to get Drupal to perform as a social network. That is the main large example of a large social network on Drupal, and it is hacked to death. What is the point of hacking something to death because of its design limitations? The amount of time hacking Drupal could have been used creating something more efficient from the ground up. With Drupal, I feel like I would have to recreate the wheel anyway while having to deal with all the mud that is stuck to it.

Anything ‘can’ scale if you throw a bunch of money at it, but it does not make it the best solution, or the most efficient solution.

I looked on drupalsites.net, and drupalsn.com to find any very high traffic sites using Drupal, did not find any examples that led me to believe that Drupal is the best solution. All those people reviewing code for Drupal are doing so and evaluating it as a content management system, not as a high traffic social network. If they did, it would be designed differently. Drupal is designed to fit a number of use cases, not just one.

Drupal is designed to do a lot of things, but I don’t think this is one of their strong point for a high traffic multiple logged in user site. Drupal has its place. After doing much research, and looking at the responses here, I don’t think this is a good situation to use Drupal at all. Drupal is not the end all be all solution for every situation. If I thought Drupal could do what I need, I would be a happy camper. It would mean faster to market, and easier to get up and running, but my research is not coming out with Drupal being the best long-term solution.

Can you name one very high traffic social network that is based on Drupal? I am not talking about sites where most of the pages are for anonymous users. I am talking about sites with numerous logged in users. I have looked extensively for weeks, and cannot find any cases of it, except for teamsugar. I have found some nice looking Drupal social networks, but I have not found very high traffic sites, and the one that is, is not really even Drupal anymore because of all the rewrites.

Two examples of Drupal running more than an average amount of ‘authenticated’ users:

[LIST=1]
[]http://www.ballardnewstribune.com/
[
]http://www.westseattleherald.com/
[/LIST]They are designed by a company in the Seattle area called Freelock - www.freelock.com - they use Drupal for many of their projects. The owner is very approachable so you might want to get his opinion off twitter @freelock.

You are right, if you have to hack the heck out of it to scale to your needs then you are better off designing your own that is aimed at the scalability you expect. Often times people believe that they can ‘write something better’ but get caught by lack of knowledge and experience in these areas which often leads to ‘lots of extra time’ and a marginal if any betterment in scalability. Your abilities may be up to this task however?

Steve

Thanks, ServerStorm.

At this point, I am just thinking about what is the best way to go about the project. I don’t think it terms of better than Drupal, just something different and for a specific purpose. I have no illusion that this is something trivial or easy or that I am the greatest programmer in the world. I guess I am looking for a challenge, and Markus Frind’s story about Plentyoffish is pretty amazing, not that I will ever build anything with that level of success. This is a niche that I care a great deal about, and I want to build the best tool I can for that niche. It is really that simple, what is the best way to build the best tool for my audience? I really wanted Drupal to be the solution, but the research has not proven that out. I have been a BIG fan of Drupal for the last few years.

@hipspy

In post #15 LogicEarth is on the track were I would invest my time/money. You know from your research that Drupal doesn’t stand up to many authenticated users before it bogs down; then by load balancing/clustering can bail you out. When needed, you scale your load balancing/clustering rather than trying to build a framework that performs like needed on a single host.

Amazon for example just launched the DynamoDB for Internet Scale Applications. Take a look at “All Things Distributed

Steve

ServerStorm, I never indicated that I was going to try and run this on a single server, and I don’t want to use hardware to compensate for Drupal’s flaws (poor database design (CCK) and very poor query design). A site like this of course will not be able to run on a single server, if it became popular. One server has never been a consideration for a long-term solution, and clustering and load balancing are required no mater which way I go. The more efficient the underlying application, the lower the requirements for the size of the various clusters, and the more money that can be saved long term. As I said in an earlier post, I don’t want to throw money at it to compensate for something that does not have the best design for a high performance website in the first place.

Related to Drupal, I don’t want something where I need to scale out to bail me out because of underlying flaws in the way that Drupal’s database and queries are designed (the teamsugar.com case study on drupal.org discusses this at length: http://drupal.org/node/116578). I don’t think LogicEarth was saying to try and use load balancing and clustering to compensate for Drupal. I think he was saying that there are other things to consider besides the application language, cms or framework to create a high performance website. In this thread, I am focused on the application, not server technologies. Clustering and such is a whole other topic way beyond what probably needs to be discussed here. In fact, LogicEarth gave language options like php, ruby and java. I don’t see anywhere in his post where he is advocating Drupal, and using server technology to bail it out.

ServerStorm, I am checking out the link you attached.

@hpspy

I did not reference LogicEarth to reinforce the idea; I was added my view on why I would invest in this rather than software development. In my experience it is the far less expensive path.

If you know that Drupal is not going to work for you then like an earlier posted said, all the CSM/Frameworks have their tradeoffs so you will be running into them if you choose one other than Drupal; your research will likely turn up similar results.

I did not mean to make assumptions regarding what you are or are not going to do, again I was basing my preference to rely more on the clustering/load balancing because in the past I invested in software development and it cost me 3 times what investing in hardware did.

I hope that you find an appropriate solution.

Steve

Thanks again, ServerStorm (Steve).

That is a good insight and a different perspective. How did the project turn out where you invested in software development? I definately agree that all of the CMSes have their tradeoffs. I was even looking at dotnetnuke, Modx, and Umbraco. I have deployed Joomla in the past and don’t want to go there again. I agree that all of them have compromises. Version 2.5 of Joomla came out yesterday I think. I’ve had a five year love affair with Drupal, but I think I am falling out of love. When I stared this thread, I wanted to get other peoples insights. As the days have passed, and I’ve gotten more insights, it seems that I have to move on from Drupal. It is not without some regrets. :blush:

When I started the thread I was hoping to find proof that Drupal was up to the job, but after doing much more research (reading the teamsugar.com case study, and numerous posts online), it does not escape me how funny it is that here I am now pointing out why Drupal is not the solution. What a difference a few days make…

A club membership system we developed started small, we rolled our own solution with the idea and planning that it could grow. As the system grew we started to get database performance issues not because of the way we coded but due to concurrency. We therefore invested software development in threading, caching, and shifted to split db responsibilities (re-wrote code to make use of this). The system grew over 6 years and got to the point that it had 1 million plus concurrent users, which is peanuts for today, but was a lot back then. Over this we had staff turnover, new people had to get familiar with the custom framework (It started with a single developer, but in the end 12 developers were employed for maintenance and updates). The staff turnover was expensive. The efficiency of the developers code was not always consistent, so as our framework grew it became slower and slower. We eventually had to scrap it a move to a different solution; we used Symphony, need only 3 developers, and invested in hardware clustering. Software costs shrunk and the additional hardware/bandwidth costs were only a fraction of our previous developer costs.

Symphony has its limitations but it worked for what we were doing.

Steve

Thanks, Steve.

Wow, you have a wealth of knowledge. Have you ever tried Yii? Thank you for taken the time to give more background for your perspective.

Rather than Symphony, if you were to say, rebuild the framework from scratch given where the current system was - would you still say that Symphony was what made the difference, or simply the ability to start from scratch rather than continuing to hack apart?

I agree with wonshikee. It was likely the simple act of starting from scratch that made it better. You were able to declutter the things that likely changed over the years.

There are lots of benefits to frameworks and CMSes, regardless which you use. They can speed up development tremendously by having lots of common elements already built in.

However, most of them have to have a certain level of generic-ness to them in order to be able to help all those people. That generic-ness is what keeps them only at “high” performance and not “ultra high” performance. There is stuff you just don’t want or need in there. Hence why those that hacked Django to death got it working… they went in with a sledgehammer and removed what they didn’t need.

Building from scratch will let you get exactly what you need, with nothing you don’t.

Both of them can become slow and difficult to maintain, depending on the developers writing the code. Bad developers write bad code, regardless what they are working in.

The learning curve is a good point and something you may run in to. However, with good documentation and a well thought out and fairly intuitive interface, you can make your framework quite easy and quick to learn.

While it’s great that the OP is doing research, always remember - any popular website has been rebuilt many times. There is simply no way for you to foresee the direction it will go, so don’t go into this with the idea that it will be perfect from the beginning and will scale smoothly.

Choose, what seems like a logical starting point and just jump in. There is no such thing as a perfect platform/framework - they all have pluses and minus - and in the end - the skill level of the developer can easily turn an asset into a liability and vice versa.

If you have the budget and ability to program I would highly recommend using a framework over a CMF/CMS. CMS/CMF are clunky, littered with poor decisions and can become pain to maintain and scale. I maintain several Drupal sites and all I ever see is things that make we want to cry from serialized data storage, virtual foreign keys, redundant queries and the list goes on. Though it is not so much Drupal itself as it is the modules that just about every install is going to have like views and panels which are a complete nightmare from data storage (did I mention serialized data) to inefficient queries. Having said that is one of the most powerful solutions available for creating a website to mini application without writing any code. So if you can’t program Drupal isn’t to bad just hope you don’t install the wrong module because when it works it works like a charm and when it doesn’t well… that is a whole other story.