Comparing Frameworks

PHP
1

When we want to compare frameworks, more often than not we do so using a Hello, World benchmark. After all, they’re simple, objective, and measurable. But the Hello, World benchmark misses the vast majority of features that a framework provides.

Recently I went looking for comparisons of JavaScript templating solutions, and I found this little gem. I found this interesting because of the way they went about evaluating their choices:

The Test

To pick between the 18 templating options, we assigned each option to a small team of front-end engineers and setup a test: render a (very simplified) LinkedIn profile.

…

The List

We gave each team a list of features to look for in their assigned templating solution. The idea was to fill out a score, from one (poor) to five (excellent), for each item:

[list]
[] DRY: how DRY is the templating technology? is there support for code-reuse and partials?
[] i18n: is there support for translations and multiple languages?
[] Hot reload: are changes visible immediately or is there a compile/deploy cycle?
[] Performance: how long does it take to render in the browser and server?
[] Ramp-up time: how is the learning curve?
[] Ramped-up productivity: once you’ve ramped-up, how fast can you build things?
[] Server/client support: can the same template be rendered both client-side and server-side?
[] Community: is there an active community using this project? Can you google issues?
[] Library agnostic: are there dependencies on other JS libraries, such as jQuery or Mootools?
[] Testable: how hard is it to write unit and integration tests?
[] Debuggable: is it possible to step through the code while it’s running to track down errors?
[] Editor support: is there an editor with auto-complete, syntax highlighting, error checking, etc?
[] Maturity: is this a relatively stable project or still experimenting and churning?
[] Documentation: how is the documentation?
[*] Code documentation: do the templates encourage/require documentation/comments?[/list]

I think this list of features for evaluation is phenomenal. If ever we were to do a thorough comparison of the various frameworks available, I imagine the list for evaluation would look very similar to this.

P.S. If anyone is interested in organizing a thorough framework comparison along these lines, then feel free to drop a quick reply saying so. If enough people are interested, this could be doable.

2

I recently wrote a blog post on how I chose a framework and what I look for (link to my dev blog in my sig below).

I follow a 2 step approach. First I make a list of minimum requirements that have to be met. In this case mine were following:

  1. PHP 5.3
  2. Ease of use
  3. Good documentation
  4. Well written code (like the code should be written sensibly & not slapped together on a weekend after 3-4 tequilas)
  5. Performance (this again is subjective - it doesnt need to be fastest but shouldn’t be something that crawls)
  6. Good ORM (kinda like a bonus point - a well integrated ORM is better than having none in which case you’ll need to work to integrate one)

So after the candidates are filtered by these set of minimum requirements, then it gets more subjective on filtration (as everyone’s needs are different).

  1. Flexibility - Is the framework somewhat flexible or rigidly imposes its rules on developer (like file names, class names etc). Can 3rd party libraries or your own previously written libraries be integrated easily or does it need them in a specific format?
  2. What’s in the box - What kind of components does it come with? If I need a Form builder or HTML builder then does it come with them? What about state management? Caching?
  3. Template Engine - Does it come with a template engine? Whats the learning curve, benefits & performance cost? Is it compulsory to use it?
  4. Readability - Is the framework API readable? Is it like:

    $q = $db->col( "SELECT id FROM users WHERE role='subscriber' ORDER BY date DESC LIMIT 100" );


    or like:

    $rs = User::where( 'role', '=', 'subscriber' )->orderBy( 'date', 'DESC' )->take( 100 )->get( array( 'id' ) );
  5. Testable - Is the framework testable? Can you write unit tests for it? Is it easy or a pain in the a**?
  6. Debuggable - Can you debug code, step through it easily using an IDE (not one specific or endorsed by framework but a generic PHP IDE with xdebug)
  7. Community - Does the framework has good community? If you get stuck somewhere then can you easily ask a question about it somewhere? Are there any good learning resources (besides the docs)?
  8. Style - Do you like the code writing style? App file/dir structuring? (yes its totally subjective)

Its not an exhaustive list by any means and the priorities might vary from individual to individual. Like someone might weigh in heavily on a framework that supports i18n out of the box while some might not since they dont need i18n.