Discourse and other boogey men [bluepill]

I thought I would do some research. It came about when I started to wonder what bluepill is. As I may have mentioned in another post that I have used rails before but a long time ago. It was version 1.x back then. Much has changed since then … or has it?

Back in the version 1.x days I was working on a rails project for a client. We deceloped and tested in Linux and all seemed to go smoothly, well mostly anyway. One day the client suddenly dropped on us that he wanted to deploy to a windows server. Well Apache works on both and so does ruby so it should work, or so we thought, ha ha. What a nightmare! The apache module for rails gave us no end of trouble and was officially unsupported on windows.

The closest we could get to a solution was to get Apache to reverse proxy to a mongrel server and mongrel was not recommended as a production server in those days. Besides, it felt wrong saying we were running rails in Apache if it was only acting as a proxy.

We ended up taking a huge risk and rewrote the project in python/Django which was not considered as advanced as rails in those days. Probably due to it being a late starter in that space. However I was pleasently surprised about how much easier it was for me to write in python/Django and it wasn’t long before we were comfortable with Django. We then deployed to windows/Apache, using mod_python in those days, and everything just worked. What a champagne moment that was! (Well it would have been if we had any :wink: )

Now of course rails has moved to v3.x with v4 coming soon. Django has gone through several iterations since then also. However I’m still not brave enough to try again and deploy a ruby on rails project on windows :wink:

Anyway, so even though I have used Rails before I had never heard of bluepill. There is so much new stuff around rails that I’ve scarcely heard of from my v1 days. I became curious about it when I saw it being talked about on meta.discourse.org.

Well, much to my surprise I found it was for restarting processes that have prematurely terminated. Basically it sits in memory monitoring other processes and if they terminate it restarts them. What processes could this be with Discourse? I am so used to only needing a web server, a database server and a scripting language when writing web applications. I’ve seldom had to restart those and the only times I’ve had to is when I’ve forgotten to run “chkconfig” after installing them. I did have a problem with mysql not starting once but it was due to shutting the machine down suddenly and an orphaned lock file. After deleting the stale lock file I could start it normally.

So anyway, with Discourse, what is bluepill starting then? Turns out its starting something called thin. Thin, as it turns out, is a web server. The boogey man of reverse proxying returns!!! This is in their instructions for an Apache configuration and now they are running Thin! Huh?? Could this be the reason why Discourse needs or suggests more ram than any other forum in common usage?

So I say to Sitepoint, when you open this “pandora’s box” you are not only getting a rails framework with coding for a forum. You are also getting whatever else is lurking in that box. Bluepill is watching your Thin processes but who is watching Bluepill? :wink:

I’m really not sure why this is posted here instead of on Discourse’s website… Really it is more of a question for them, one they can give you a better answer on. However, if it puts your mind at ease, Discourse will be hosting the forum. We simply will have a few A records to update so they can use the subdomain we have laid out for it. With that being said, the only thing it will ever see, is Discourse related… so I don’t see much of a worry, but then again, I couldn’t answer your question about bluepill to begin with (Discourse could though).

Ok, just that I thought Sitepoint staff were rails devs and therefore would make perfect sense to you.

As to why it wasn’t posted on Discourse site was because I wasn’t aware bluepill was the sole domain of Discourse development. I assumed it was a normal part of rails development that I had missed out on in my absence from rails.

Interesting that it’s hosted. I wonder what kind of access that gives you to your own plugins and own development.

I am actually wondering where this statement is coming from. I thought Sitepoint was looking to widening its engagement from just PHP and SEO talk. Ok, maybe I should have posted this to the rails forum but it was hard to know. It was topical and relevant to right now so I posted it here.

Incidentally I don’t ask too much on Discourse Meta as they either tend to ignore me or fob me off. Like the question I asked them for a breakdown of RAM usage was fobbed of with a meaningless response of “awesomeness needs more RAM”. I suspect they are following StackOverflow’s adage of “suppressing discussion” and only giving short answers.

I just assumed you guys might be interested in Discourse internals seeing you are looking to use it. Perhaps I guessed wrongly.

Keep in mind that with the exception of HAWK the rest of the forum Staff are volunteer forum members not SitePoint employees.

While many of us might be considered programmers, many other’s area of interest is in other areas, mark-up, server management, marketing etc.
Individually none of us is strong in all areas but collectively I think we do well.
So though some of us can write Ruby, it is the SitePoint employed developers that will be in charge of the back end stuff. Our role is the face of the forums.

Although the best answers about how Discourse is doing things would come from them, I think the best chance for getting bluepill answers here would be the Ruby & Rails forum, so I’ll move it over now.

Sorry, maybe I mis-understood your questions or mis-read the verbiage, but I came off of it thinking you were asking how bluepill affected Discourse and any other process thereof on the server (with more emphasis on Discourse). Which is why my recommendation was to ask Discourse (they could tell you better than anyone, even if I studied their source for a few days).

As for bluepill in general, I’d have to see the Discourse code, but from this article’s example, it seems you give it the process name and it watches that process. Now that article was likely a few years ago, as its ruby version is a little far back, so I found these articles too (http://help.cloud66.com/how-to/bluepill.html, and SO http://stackoverflow.com/questions/18860985/how-to-daemonize-a-rails-script-with-bluepill).

Since none of those really satisfied my curiosity, I went to the GitHub for bluepill and found in the commands, that it really does work within an application. So with its field of vision limited to the application, I don’t really see a lot to worry about (which is visible by looking at the commands, each ends with “for an application”).

Also, I’m not a ruby expert (or rails expert). In fact, I only taught myself some basic Ruby a couple of weeks ago. I’m more of a .NET/PHP guy and I lean harder on .NET than PHP anymore. So all of the above is from my research on Bluepill and not from any experience using it.

Thanks @Mittineague for moving thread over to correct forum.

I don’t agree that Bluepill does work “in an application” seeing the documentation says it runs as a daemon (or at least as its own process), I know Bluepill itself doesn’t use much memory according to the reports.

What I find disturbing is Jeff Atwood’s comments on Meta that implied that Discourse has no bloat. Now I find the standard install is running secondary web servers in their own memory space. Ok, I get running cron jobs to run code that doesn’t naturally fit into the http request/response cycle but secondary webservers I go “huh?”. I have to wonder what the Discourse dev team are thinking.

I really am starting to wonder if the Discourse management don’t care about how easy or difficult this is to install or whether they rule out shared hosting options like GoDaddy and Hostgator altogether. Their chances of producing a “1 click installer like wordpress” are galloping like wild horses towards the horizon maybe never to be seen again. I guess they are thinking more $$$ for them. Its an ingenious business strategy.

Read it again… (emphasis mine)

CLI

Usage

bluepill [app_name] command [options]
For the “load” command, the appname_ is specified in the config file, and must not be provided on the command line.

For all other commands, the appname_ is optional if there is only one bluepill daemon running. Otherwise, the appname_ must be provided, because the command will fail when there are multiple bluepill daemons running. The example commands below leaves out the appname_.

It seems to be application controlled. Otherwise, you wouldn’t need to provide an app name in a config file or pass it in… Plus since multiple bluepills can run on the same server, it seems even more apparent that it is app driven. Just my take from the documentation.

No, I still don’t agree. Maybe it can be used that way but that’s not the way Discourse is using it. If you look at their installation documentation they are booting up bluepill from rvm. No parameters are passed, they are all read from a configuration file (or a configuration-like file because it looks like ruby script but without the .rb extension).

I am referring to the Discourse installation documentation above, not the bluepill documentation (just to try and avoid confusion).

Can you point me to the files in question?

https://github.com/discourse/discourse/blob/master/config/discourse.pill.sample

Also the following under section “Bluepill setup”
https://github.com/discourse/discourse/blob/master/docs/INSTALL-ubuntu.md

It seems from that “bluepill” is being run from “rvm”, not executed from a ruby (.rb style) application via any of the usual module imports.

Cool, added to my list to look at today (once I get some caffeine running through my veins). Hopefully, once we are on the same page, I will either 1) see what you are referring to, or 2) spot something that may make more sense (I’d be happy to find either one)

https://meta.discourse.org/t/free-hosted-option/9621

Even though the title of this thread is not really in line with my threads discussion, it does shed light on what those Thin servers are and that they are indeed multiplied for each “configuration instance” and places strong upper limits on the number of Discourse instances a single machine can host. Hence it does in fact seem like Discourse is a “resource hog”.

Huh? did you post the right link? If so. which post are you referring to?

Yep, but it was only post #13 by McKay that was relevant. He explains a bit on how Discourse scales to multiple customers on the same host. (In other words it doesn’t particularly.)

I still don’t get it, it looks to be the same call that is on bluepill GitHubs repo.

Bluepill.application("discourse", :base_dir => ENV["HOME"] + '/.bluepill') do |app|

“discourse” is the application name, so it is being provided. It looks like it is using the rvm to read environment data, not start Bluepill. Bluepill has to already be started for you to be able to use it within your application, you are simply telling Bluepill what processes you want watched that are related to your application (sidekiq and thin for example).

Granted, I still don’t know ruby all that well, but that is how it reads to me…

I won’t argue with you on that statement, the high memory/cpu requirements indicated that from the beginning, but Jeff has also openly stated, that their focus is on getting core items developed fully first, then to go back and clean up code to make it perform better on lower requirements (granted, we all know the likelihood of the latter happening is little to none, as what company would want to spend money with no visible improvement/features to show for it?)

Well, I suspect the founders of StackOverflow woke up one morning and said “Our high page ranking .net platform is looking a bit old and worn. Its become difficult to extend, and it really needs a rewrite. What can we do?”

“I know”, said another, “let’s see if we can turn this into a PR exercise. Let’s rewrite this platform in RoR, make it open source, be sure to make it bloated enough so it doesn’t run on garden variety shared hosting, and start our own hosting offering for $$. People will surely flock to us with their credit cards and we will be seen as the saviours of the internet.”

Quite a brilliant marketing move I think. However I do wonder if Jeff Atwoods is the new Michael “Monty” Widenius (MariaDB) or the old Larry Ellison (Oracle/MySql).

^^^ Not quite. My understanding is Jeff left that bunch and moved on to the next project because StackOverflow was done from a “dream up new venture / app” standpoint. Joel is still flogging SO and didn’t come over to discourse.

As for rails on windows I’ve done it (IIS AAR + mongrel). App wasn’t particularly busy but it stood up pretty well. Biggest issue we had was the “snowflake” deployment, was difficult and tricky to repeat. Reverse proxies in general are a wonderful thing and they make tons of sense for non-php style apps where you’ve got a persistent app container that serves the app and doesn’t handle static files so well as well as all that bad stuff that happens on the internet that leads one to need a hardend IIS / Apache / Nginx setup facing the world.