Paul M. Jones

Don't listen to the crowd, they say "jump."


New Year's Benchmarks

After the previous round of benchmarking, I received one very good criticism from Matthew Weier O'Phinney about it. He suggested that the hardware I was using, a PowerPC G4 Mac Mini, had an I/O system that was not representative of what a "regular" user would have as a web server. I have to agree with that.

As such, I have prepared a new set of benchmarks for Cake, Solar, Symfony, and Zend Framework using an almost identical methodology as last time.

Much of this article is copied directly from the previous one, as it serves exactly the same purpose, and little has changed in the methodology. Please consider the older article as superseded by this one.

Introduction

If you like, you can go directly to the summary.

This report outlines the maximum requests-per-second limits imposed by the following frameworks:

The benchmarks show what the approximate relative responsiveness of the framework will be when the framework's controller and view components are invoked.

Full disclosure: I am the lead developer of one of the frameworks under consideration (Solar), and I have contributed to the development of another (Zend Framework). I have attempted not to let this bias my judgment, and I outline my methodology in great detail to set readers at ease concerning this.

Why These Particular Frameworks?

In my opinion, the frameworks in this comparison are the most full-featured of those available for PHP. They make good use of design patterns, provide front and page controllers, allow for controller/view separation, and offer an existing set of plugins or added functionality like caching, logging, access control, authentication, form processing, and so on.

Some Code Igniter folks left comments previously asking that I include CI in the benchmark process. I don't mean to sounds like a jerk, but after reviewing the CI code base, it is my opinion (note the small "o") that it is not in the same class as Cake, Solar, Symfony, and Zend. I won't go into further detail than that; I think reasonable and disinterested observers will tend to agree. Code Igniter is indeed very fast, but as I note elsewhere, there is more to a framework than speed alone.

Regarding frameworks in languages other than PHP, I don't have the expertise to set them up for a fair comparison. However, you can compare the results noted in this report with this other report. Perhaps by extrapolation, one can estimate how Rails and Django compare to the other PHP frameworks listed here.

Methodology

For each of the frameworks tested, I attempted to use the most-minimal controller and action setup possible, effectively a "Hello World!" implementation using the stock framework components and no configuration files (or as few as the framework would let me get away with). This was the only way I could think of to make sure the tests were identical on each framework.

The minimalist approach measures the responsiveness of the framework components themselves, not an application. There's no application code to execute; the controller actions in each framework do the least possible work to call a view. This shows us the maximum possible throughput; adding application code will only reduce responsiveness from this point.

In addition, this approach negates most "if you cache, it goes faster" arguments. The "view" in each case is just a literal string "Hello World!" with no layout. It also negates the "but the database connection speed varies" argument. Database access is not used at all in these benchmarks.

Server

Previously, I ran the benchmarks on a Mac Mini G4; as noted by others, it is not exactly a "real" production server. With that in mind, these new benchmark tests were run on an Amazon EC2 instance, which is a far more reasonable environment:

  • 1.7Ghz x86 processor
  • 1.75GB of RAM
  • 160GB of local disk
  • 250Mb/s of network bandwidth.

Clay Loveless set up the instance with ...

  • Fedora Core 5
  • Apache 2.2 and mod_php
  • PHP 5.2.0 with Xcache (64M RAM)

Setup

Each framework benchmark uses the following scripts or equivalents ...

  • Bootstrap file
  • Default configuration (or as close as possible)
  • Front-controller or dispatcher
  • Page-controller or action-controller
  • One action with no code, other than invoking a View processor
  • Static view with only literal text "Hello World!"

... so the benchmark application in each case is very small.

I did not modify any of the code in the frameworks, except for one condition. I modified the bootstrap scripts so they would force the session_id() to be the same every time. This is because the benchmarking process starts a new session with each request, and that causes responsiveness as a whole to diminish dramatically.

Benchmarking Tools

I wrote a bash script to automate the benchmarking process. It uses the Apache benchmark "ab" tool for measuring requests-per-second, on localhost to negate network latency effects, with 10 concurrent requests for 60 seconds. The command looks like this:

ab -c 10 -t 60 http://localhost/[path]

The benchmark script restarts the web server, then runs the "ab" command 5 times. This is repeated for each version of each framework in the test series, so that each gets a "fresh" web server and xcache environment to work with.

Benchmark Results

Baseline

First, we need to see what the maximum responsiveness of the benchmark environment is without any framework code.

framework 1 2 3 4 5 avg
baseline-html 2613.56 2284.98 2245.98 2234.94 2261.01 2328.09
baseline-php 1717.74 1321.49 1292.86 1511.40 1327.35 1434.17

The web server itself is capable of delivering a huge number of static pages: 2328 req/sec for a file with only "Hello World!" in it.

Invoking PHP (with Xcache turned on) to "echo 'Hello World!'" slows things down a bit, but it's still impressive: 1434 req/sec.

Cake

I ran benchmarks for Cake with the debug level set to zero. I edited the default bootstrap script to force the session ID with this command at the very top of the file...

<?php
session_id('abc');
?>

... but made no other changes to the distribution code.

The page-controller looks like this:

<?php
class HelloController extends AppController {
    var $layout = null;
    var $autoLayout = false;
    var $uses = array();
    function index()
    {
    }
}
?>

The page-controller automatically uses a related "index.thtml" view.

Running the benchmarking script against each version of Cake gives these results:

framework 1 2 3 4 5 avg
cake-1.1.10 85.65 85.87 85.71 85.66 85.93 85.76
cake-1.1.11 113.78 114.13 113.59 113.35 113.73 113.72
cake-1.1.12 114.62 114.26 114.55 113.89 114.64 114.39

So the most-recent release of Cake in a fully-dynamic mode has a top-limit of about 114 requests per second in the benchmarking environment.

Solar

The Solar bootstrap file looks like this; note that we force the session ID for benchmarking purposes.

<?php
session_id('abc');

error_reporting(E_ALL|E_STRICT);
ini_set('display_errors', true);

$dir = dirname(__FILE__) . DIRECTORY_SEPARATOR;

$include_path = $dir . 'source';
ini_set('include_path', $include_path);

require_once 'Solar.php';

$config = $dir . 'Solar.config.php';
Solar::start($config);

$front = Solar::factory('Solar_Controller_Front');
$front->display();

Solar::stop();
?>

The Solar page-controller code looks like this:

<?php
Solar::loadClass('Solar_Controller_Page');
class Solar_App_HelloMini extends Solar_Controller_Page {
    public function actionIndex()
    {
    }
}
?>

The page-controller automatically uses a related "index.php" file as its view script, and uses no layout.

Running the benchmarking script against each version of Solar gives these results:

framework 1 2 3 4 5 avg
solar-0.25.0 170.80 169.22 170.29 170.75 170.22 170.26
solar-svn 167.83 166.97 167.56 164.56 162.30 165.84

Clearly the current Subversion copy of Solar needs a little work to bring it back up the speed of the most recent release, but it can still handle 165 requests per second in the benchmarking environment.

Symfony

The Symfony action controller code looks like this:

<?php
class helloActions extends sfActions
{
  public function executeIndex()
  {
  }
}
?>

The action controller automatically uses a related "indexSuccess.php" file as its view script. However, there is a default "layout.php" file that wraps the view output and adds various HTML elements; I edited it to look like this instead:

<?php echo $sf_data->getRaw('sf_content') ?>

If there is some way to make Symfony not use this layout file, please let me know.

Finally, I added session_id('abc'); to the top of the web/index.php bootstrap script to force the session ID to be same for each request.

Running the benchmarking script against each version of Symfony gives these results:

framework 1 2 3 4 5 avg
symfony-0.6.3 53.10 53.15 52.27 52.12 52.10 52.55
symfony-1.0.0beta2 67.42 66.92 66.65 67.06 67.83 67.18

It looks like the most-recent beta version of Symfony can respond to about 67 requests per second in the benchmarking environment.

Zend Framework

As before, the Zend Framework involves some extra work. As others have noted, Zend Framework requires more putting-together than the other projects listed here.

The Zend 0.2.0 bootstrap script looks like this ...

<?php
session_id('abc');

error_reporting(E_ALL|E_STRICT);
ini_set('display_errors', true);
date_default_timezone_set('Europe/London');

$dir = dirname(__FILE__) . DIRECTORY_SEPARATOR;

set_include_path($dir . 'source/library');

require_once 'Zend.php';
require_once 'Zend/Controller/Front.php';

Zend_Controller_Front::run($dir . 'application/controllers');
?>

... and the Zend 0.6.0 bootstrap looks like this:

<?php
session_id('abc');

error_reporting(E_ALL|E_STRICT);
ini_set('display_errors', true);
date_default_timezone_set('Europe/London');

$dir = dirname(__FILE__) . DIRECTORY_SEPARATOR;

set_include_path($dir . 'source/library');

require_once 'Zend.php';
require_once 'Zend/Controller/Front.php';

$front = Zend_Controller_Front::getInstance();
$front->setControllerDirectory($dir . 'application/controllers');
$front->setBaseUrl('/zend-0.6.0/');

echo $front->dispatch();
?>

The differences are minor but critical, since the front-controller internals changed quite a bit between the two releases.

Similarly, the page-controller code is different for the two releases as well. Zend Framework does not initiate a session on its own, whereas the other frameworks do. I think having a session available is a common and regular requirement in a web environment. Thus, I added a session_start() call to the page-controller for Zend; this mimics the session-start logic in the other frameworks. Also, the Zend Framework does not automatically call a view, so the page-controller code below mimics that behavior as well.

The page-controller code for Zend 0.2.0 looks like this ...

<?php
require_once 'Zend/Controller/Action.php';
require_once 'Zend/View.php';
class IndexController extends Zend_Controller_Action
{
    public function norouteAction()
    {
        return $this->indexAction();
    }

    public function indexAction()
    {
        // mimic the session-starting behavior of other application frameworks
        session_start();

        // now for the standard portion
        $view = new Zend_View();
        $view->setScriptPath(dirname(dirname(__FILE__)) . '/views');
        echo $view->render('index.php');
    }
}
?>

... and the page-controller code for Zend 0.6.0 looks like this:

<?php
require_once 'Zend/Controller/Action.php';
require_once 'Zend/View.php';
class IndexController extends Zend_Controller_Action
{
    public function indexAction()
    {
        // mimic the session-starting behavior of other application frameworks
        session_start();

        // now for the standard portion
        $view = new Zend_View();
        $view->setScriptPath(dirname(dirname(__FILE__)) . '/views');
        echo $view->render('index.php');
    }
}
?>

Running the benchmarking script against each version of Zend Framework gives these results:

framework 1 2 3 4 5 avg
zend-0.2.0 215.91 208.50 207.84 210.43 211.73 210.88
zend-0.6.0 133.60 134.18 132.10 129.53 130.16 131.91

In the benchmarking environment, the most-recent Zend Framework can handle about 131 requests per second.

But look at Zend 0.2.0 -- 210 req/sec! Richard Thomas noted this in a comment on the last set of benchmarks. If that's a sign of what a future release might be capable of, it's quite stunning.

Summary

To compare the relative responsiveness limits of the frameworks, I assigned the slowest average responder a factor of 1.00, and calculated the relative factor of the others based on that.

framework avg rel
cake-1.1.10 85.76 1.63
cake-1.1.11 113.72 2.16
cake-1.1.12 114.39 2.18
solar-0.25.0 170.26 3.24
solar-svn 165.84 3.16
symfony-0.6.3 52.55 1.00
symfony-1.0.0beta2 67.18 1.28
zend-0.2.0 210.88 4.01
zend-0.6.0 131.91 2.51

Thus, the Zend 0.2.0 framework by itself is about 4 times more responsive than Symfony 0.6.3. (My opinion is that this is likely due to the fact that the Zend code does less for developers out-of-the-box than Symfony does.)

For comparison of the currently-available options, we can highlight the most-recent releases of each framework, and place them in order for easy reading:

framework avg rel
solar-0.25.0 170.26 3.24
zend-0.6.0 131.91 2.51
cake-1.1.12 114.39 2.18
symfony-1.0.0beta2 67.18 1.28

As we can see, the framework with the greatest limits on its responsiveness is Symfony. Zend is about 15% faster than Cake. And as with the last benchmark series, Solar has the least limit on its responsiveness in a dynamic environment.

You can download the individual benchmark log files, including the report summary table, here.

Conclusion

It is important to reiterate some points about this report:

  • The benchmarks note the top limits of responsiveness of the frameworks components themselves. This functions as a guide to maximum speed you will be able to squeeze from an application using a particular framework. An application cannot get faster than the underlying framework components. Even "good" application code added to the framework will reduce responsiveness, and "bad" application code will reduce it even further.

  • Each of these frameworks is capable of building and caching a static page that will respond at the maximum allowed by the web server itself (about 2300 requests/second, as noted in the baseline runs). In such cases, the responsiveness of the framework itself is of no consequence whatsoever. But remember: the moment you hit the controller logic of a framework, responsiveness will drop to the levels indicated above.

  • Dynamic responsiveness is only one of many critical considerations when choosing a framework. This benchmark cannot measure intangibles such as productivity, maintainability, quality, and so on.

Again, it is entirely possible that I have failed to incorporate important optimizations to one or more of the frameworks tested. If so, it is through ignorance and not maliciousness. If you are one of the lead developers on the frameworks benchmarked here and feel your project is unfairly represented, please contact me personally, and I'll forward a tarball of the entire benchmark system (including the framework code as tested) so you can see where I may have gone wrong.

UPDATE (2007-01-05): I have created a repository of the benchmark project code for all to examine.


The Interesting Case of Diminishing Responsiveness

I'm working up a second run of PHP framework benchmarks on more powerful hardware than the last time. I'll talk more about that in a future entry, but in the mean time, I wanted to share an experience that may help you out.

I was running the ab (Apache Benchmark) tool against Zend Framework, Solar, and Cake, using the minimalist "hello world" applications I described the previously. The ZF requests/minute were consistent between runs, always "Z" amount (plus or minus a couple percentage points).

But the Solar and Cake requests/minute declined considerably from run to run. For example, the first benchmark run would yield "X" requests/minute; the 2nd run would yield 0.93X, the 3rd 0.85X, then 0.75X, 0.64X, 0.52X, and so on. This was with the opcode caching turned on; without it, the decline was less severe, but it was still there. Very depressing.

Because I know Solar better than I know Cake, I started looking around the codebase for places to improve. Was it the way I was loading classes, thus causing the opcode cache to work inefficiently? Was it a memory leak in Solar, or PHP itself, or maybe in the web server somehow?

Nope: it was sessions.

Solar and Cake each call session_start() on new requests. Because the "ab" tool sends a new request each time, the frameworks start a new session each time. This means that 1000 requests will generate 1000 new session files. Because the benchmarking makes thousands of requests, the /tmp directory was overflowing with sess_* files (50,000 or more). Having a very large number of files in a single directory was causing the filesystem to drag, and so response time diminished exponentially over time.

Notice that Zend Framework did not have this problem, because it doesn't have a session-management class; thus, no sessions were created, explaining why its performance stayed consistent from run to run.)

The solution (for benchmarking purposes) was to force the session ID value to ‘abc' by calling session_id('abc') command at the top of the Solar and Cake bootstrap scripts. This makes the script think there's already a session in place, and so it doesn't create a new file. Once I did that, the diminishing responsiveness disappeared, and I got consistent results from run to run.


How Fast Is Your Framework?

UPDATE (2007-01-01): I have conducted a new series of benchmarks; see them here. The numeric results below should be considered as outdated, although the history, methodology, and approach are still valid.

Wherein I relate the results of benchmark testing on three (plus one)four frameworks: Cake, Solar, Symfony, and Zend Framework(plus Cake). Anger-inducing broad-brush overview: Solar is 4x faster than Zend, and almost 2x faster than Symfony. Read on for all the nuances and caveats.

UPDATE: Have updated the Cake results, now that I've figured out how to turn off database access.

UPDATE 2: Added note about "views" in Zend Framework, also added list of loaded files and classes for each framework.

Introduction

"Figures don't lie, but liars can figure." -- Anon.

Recently I became curious how many requests-per-second the Solar framework could handle. This was related to the recent Mashery launch; we needed to be ready to receive a ton of visitors to the site (I won't disclose those numbers at this time ;-).

Most Mashery applications are built on top of the Solar framework, but when benchmarking those applications, I needed to know what the theoretical limit to responsiveness was, as imposed by the Solar framework.

After that, I figured it would be interesting to know the theoretical limits to responsiveness imposed by other frameworks, in particular Cake, Symfony, and the Zend Framework. This blog entry reports my findings.

Goals and Caveats

The purpose of this is not to say that one framework is better than another, or to engage in one-upmanship. The results do not say whether an application will run faster or slower on a particular framework; bad application code will slow you down dramatically on any framework. Similarly, use of full-page caching will bypass the framework completely, raising the responsiveness to something closer to the web-server's maximum.

Instead, the purpose of this is to tell you what the maximum possible responsiveness of the framework will be under the simplest and most straightforward conditions, while using the framework's controller and view components. Anything you add as application code will only reduce its responsiveness from there.

In addition, these results do not constitute an endorsement or indictment of any particular framework; they all have different strengths and weaknesses. I think the ones I tested are the most full-featured, in that they provide full-up MVC separation, make good use of development patterns, and offer an existing set of plugins or added functionality like caching, logging, access control, authentication, form processing, and so on.

Finally, it is entirely possible that I have missed some important speed-enhancing technique with one or more of the frameworks. If so, it is due to ignorance on my part, not maliciousness. I have done my best to honestly and fairly represent each of the frameworks here, and I would be very happy to hear what I can do to speed up any of the frameworks I benchmark below.

Methodology

For each of the frameworks tested, I attempted to use the most-minimal controller and action setup possible, effectively a "Hello World!" implementation using the stock framework components and no configuration files (or as few as the framework would let me get away with). This was the only way I could think of to make sure the tests were identical on each framework.

The minimalist approach has a number of effects:

  • It negates most "if you cache, it goes faster" arguments. The "view" in each case is just a literal string "Hello World!" with no layout wrapper. (For Symfony and Cake, this meant editing the layout files to only echo the view output.)

  • It negates the "but the database connection speed varies" argument. Database access is not used at all in these benchmarks. (I could not figure out how to turn off database access in Cake, which is why I don't include it as a peer in the results.)

  • There's no application code to execute; the controller actions in each framework are empty and only call the view, which itself is only the literal "Hello World!" text. This shows us the maximum possible throughput; adding application code will only reduce responsiveness from this point.

Technology and Setup

All the benchmark tests were run on this hardware (admittedly underpowered):

  • Macintosh Mini G4 with 1G RAM and 7200 RPM hard drive
  • Stock installation of Apache 1.3.33
  • Custom compiled PHP 5.2.0
  • APC 3.0.11 enabled with shm_size of 30 and stat turned on

Each framework benchmark uses the following scripts or equivalents ...

  • Bootstrap file
  • Default configs (or as close as possible)
  • Front-controller or dispatcher
  • Page-controller or action-controller
  • One action with no code, other than invoking a View processor
    • No authentication, no access control, no filters, no database, etc.
    • (although Cake kind of forces the database on you, so it's not a full peer in this report)
  • Static view with only literal text "Hello World!"
    • No PHP, helpers, layout, escaping, etc.

... so the benchmark application in each case is very small.

I used the Apache benchmark "ab" tool for measuring requests-per-second, on localhost to negate network latency effects, with 10 concurrent requests for 60 seconds. The command looks like this:

ab -c 10 -t 60 http://localhost/[path]

Benchmark Test Results

First, we need to see what the maximum responsiveness is without any framework code. The web server itself is capable of delivering a huge number of static pages: 808 req/sec (!!!) for a file with only "Hello World!" in it.

Document Path:          /helloworld.html
Document Length:        12 bytes
Requests per second:    808.95 [#/sec] (mean)
Time per request:       12.36 [ms] (mean)
Time per request:       1.24 [ms] (mean, across all concurrent requests)
Transfer rate:          238.69 [Kbytes/sec] received

Invoking PHP to "echo 'Hello World!'" slows things down a bit, but it's still impressive (remember, APC is turned on for all requests): 597 req/sec.

Document Path:          /helloworld.php
Document Length:        12 bytes
Requests per second:    597.61 [#/sec] (mean)
Time per request:       16.73 [ms] (mean)
Time per request:       1.67 [ms] (mean, across all concurrent requests)
Transfer rate:          107.00 [Kbytes/sec] received

Not too shabby for a consumer Macintosh Mini optimized for desktop use.

Cake

The Cake page-controller looks like this:

<?php
class HelloWorldController extends AppController {
    var $layout = null;
    var $autoLayout = false;
    var $uses = array();
    function index()
    {
    }
}
?>

The page-controller automatically uses a related "index.thtml" view script.

Respresentative result from three "ab" runs on Cake 1.1.10.3825: 17 req/sec.

Document Path:          /cake/helloworld/index
Document Length:        12 bytes
Requests per second:    17.13 [#/sec] (mean)
Time per request:       583.67 [ms] (mean)
Time per request:       58.37 [ms] (mean, across all concurrent requests)
Transfer rate:          5.09 [Kbytes/sec] received

Solar

The Solar bootstrap file looks like this:

<?php
error_reporting(E_ALL|E_STRICT);
ini_set('display_errors', true);

require_once 'Solar.php';
Solar::start();

$front = Solar::factory('Solar_Controller_Front');
$front->display();

Solar::stop();
?>

The Solar page-controller code looks like this:

<?php
Solar::loadClass('Solar_Controller_Page');
class Solar_App_HelloMini extends Solar_Controller_Page {
    public function actionIndex()
    {
    }
}
?>

The page-controller automatically uses a related "index.php" file as its view script, and uses no layout.

Representative result from three "ab" runs on Solar 0.25.0: 45 req/sec.

Document Path:          /index.php/hello-mini/index
Document Length:        12 bytes
Requests per second:    44.78 [#/sec] (mean)
Time per request:       223.32 [ms] (mean)
Time per request:       22.33 [ms] (mean, across all concurrent requests)
Transfer rate:          16.75 [Kbytes/sec] received

Symfony (Stable and Beta)

The Symfony action controller code looks like this:

<?php
class worldActions extends sfActions
{
  public function executeIndex()
  {
  }
}
?>

The action controller automatically uses a related "indexSuccess.php" file as its view script. However, there is a default "layout.php" file that wraps the view output and adds various HTML elements; I edited it to look like this instead:

<?php echo $sf_data->getRaw('sf_content') ?>

If there is some way to make Symfony not use this layout file, please let me know.

Representative result from three "ab" runs on Symfony 0.6.3-stable: 26 req/sec.

Document Path:          /symfony/web/world/index
Document Length:        12 bytes
Requests per second:    25.60 [#/sec] (mean)
Time per request:       390.70 [ms] (mean)
Time per request:       39.07 [ms] (mean, across all concurrent requests)
Transfer rate:          8.40 [Kbytes/sec] received

Representative result from three "ab" runs on Symfony 0.7.1914-beta: 24 req/sec.

Document Path:          /symfony/web/world/index
Document Length:        12 bytes
Requests per second:    23.94 [#/sec] (mean)
Time per request:       417.72 [ms] (mean)
Time per request:       41.77 [ms] (mean, across all concurrent requests)
Transfer rate:          7.86 [Kbytes/sec] received

Zend Framework (0.2.0 and incubator)

The Zend Framework bootstrap file looks like this:

<?php
error_reporting(E_ALL|E_STRICT);
date_default_timezone_set('Europe/London');

set_include_path(
    '.'
    . PATH_SEPARATOR . './incubator/library'
    . PATH_SEPARATOR . get_include_path()
);

require_once 'Zend.php';
require_once 'Zend/Controller/Front.php';

Zend_Controller_Front::run('./application/controllers');
?>

The Zend Framework action controller code looks like this:

<?php
require_once 'Zend/Controller/Action.php';
class IndexController extends Zend_Controller_Action
{
    public function indexAction()
    {
        include dirname(dirname(__FILE__)) . '/views/index.php';
    }
}
?>

Zend Framework has a couple of special cases:

  1. The front-controller can be used by itself or with a Rewrite Router. On advice of Matthew Weier O'Phinney, I used the front-controller without the router, which slows things down.

  2. Zend Framework is still in heavy development; again, on advice of Matthew Weier O'Phinney, I added the "incubator" directory to the include_path to be sure the most-recent code is being used.

  3. The Zend_View class apparently has some issues on PHP 5.2.0 that preclude its use, so I just do a plain "include" to get the view script. This is much faster than using a separate View class, but in normal use you would have a Zend_View instance here instead.

Even with that special treatment, the respresentative result from three "ab" runs on Zend Framework 0.2.0 is the lowest of the bunch: 11 req/sec.

Document Path:          /zf/index
Document Length:        12 bytes
Requests per second:    11.17 [#/sec] (mean)
Time per request:       895.28 [ms] (mean)
Time per request:       89.53 [ms] (mean, across all concurrent requests)
Transfer rate:          2.66 [Kbytes/sec] received

"This Is Stupid And Doesn't Mean Anything!"

"There are three kinds of lies: lies, damn lies, and statistics." -- variously attributed to B. Disraeli, M. Twain, and others.

"A man's got to know his limitations." -- Dirty Harry Callahan

I can hear the objections now:

  • "Not realistic!"
  • "Not comprehensive!"
  • "Doesn't account for features that I like!"
  • "Who cares, I don't need that level of responsiveness!"
  • "Doesn't matter if Framework X is slower, I'm more productive with it!"

Yes, yes, you're all correct. But when you get to the point when you have to scale across multiple servers, and you're looking to squeeze out every possible request you can, it'll be good to know what the realistic top-limit is.

Last caveat: they say "your mileage may vary." In these cases, your mileage is almost certain to vary. The absolute numbers will be different from server to server. Data and output caching can help in many cases, but the moment you invoke the front-controller and page-controller, you're going to hit these relative top limits no matter how much partial caching you use:

                  Req/Sec    % Relative
Cake              17         154%
Solar             45         409%
Symfony (avg)     25         227%
Zend Framework    11         100% (baseline)

And again, it only gets slower from there; these numbers were for the simplest possible "Hello World!" application in each framework.

Appendix: What Classes Get Loaded?

John Herren in the comments below asks how many files are loaded each time; below I give the counts as reported by get_included_files(), as well as the classes used by each framework in the benchmark application.

Cake (26)

34 files total, including these classes:

  1. Object
  2. CakeSession
  3. Security
  4. NeatArray
  5. Inflector
  6. Configure
  7. Dispatcher
  8. Router
  9. Controller
  10. Component
  11. View
  12. Helper
  13. ConnectionManager
  14. DataSource
  15. DATABASE_CONFIG
  16. Model
  17. ClassRegistry
  18. AppModel
  19. HelloWorld
  20. AppController
  21. HelloWorldController
  22. SessionComponent
  23. DboSource
  24. DboMysql
  25. HtmlHelper
  26. SessionHelper

Solar

18 files total, including these classes:

  1. Solar
  2. Solar_Base
  3. Solar_Controller_Front
  4. Solar_Uri
  5. Solar_Uri_Action
  6. Solar_Request
  7. Solar_Controller_Page
  8. Solar_App_HelloMini
  9. Solar_Session
  10. Solar_View
  11. Solar_Struct
  12. Solar_View_Helper
  13. Solar_Class_Stack
  14. Solar_Path_Stack
  15. Solar_View_Helper_GetTextRaw

Symfony (43)

42 files total, including these classes:

  1. sfConfig
  2. sfToolkit
  3. Symfony
  4. sfConfigCache
  5. sfException
  6. sfComponent
  7. sfAction
  8. sfActions
  9. sfActionStack
  10. sfActionStackEntry
  11. sfContext
  12. sfController
  13. sfFilter
  14. sfExecutionFilter
  15. sfRenderingFilter
  16. sfFilterChain
  17. sfRequest
  18. sfResponse
  19. sfStorage
  20. sfUser
  21. sfView
  22. sfWebController
  23. sfFrontWebController
  24. sfWebRequest
  25. sfSessionStorage
  26. sfPHPView
  27. sfRouting
  28. sfDatabaseManager
  29. myFrontWebController
  30. myWebRequest
  31. sfParameterHolder
  32. sfWebResponse
  33. sfBasicSecurityUser
  34. myUser
  35. sfSecurityFilter
  36. sfBasicSecurityFilter
  37. worldActions
  38. sfCommonFilter
  39. sfFlashFilter
  40. sfValidatorManager
  41. sfOutputEscaper
  42. sfOutputEscaperGetterDecorator
  43. sfOutputEscaperArrayDecorator

Zend (28)

34 files total, including these classes:

  1. Zend
  2. Zend_Exception
  3. Zend_Registry_Exception
  4. Zend_Registry
  5. Zend_Controller_Front
  6. Zend_Controller_Exception
  7. Zend_Controller_Plugin_Abstract
  8. Zend_Controller_Request_Abstract
  9. Zend_Controller_Response_Abstract
  10. Zend_Controller_Plugin_Broker
  11. Zend_Controller_Router_Exception
  12. Zend_Controller_Request_Http
  13. Zend_Http_Exception
  14. Zend_Uri
  15. Zend_Uri_Exception
  16. Zend_Uri_Http
  17. Zend_Filter
  18. Zend_Filter_Exception
  19. Zend_Uri_Mailto
  20. Zend_Http_Request
  21. Zend_Controller_Request_Exception
  22. Zend_Controller_Router
  23. Zend_Controller_Dispatcher_Exception
  24. Zend_Controller_Action
  25. Zend_Controller_Action_Exception
  26. Zend_Controller_Dispatcher
  27. Zend_Controller_Response_Http
  28. IndexController

Belated Quick Hit: Solar 0.25.0 released

I released Solar version 0.25 a week ago today, but my workload has prevented me from blogging it (and from blogging other stuff too). No major changes, just minor bugfix and minor feature enhancements; you can view the change notes here.


Solar 0.24.0 Released

Last night, I released version 0.24.0 of Solar, the simple object library and application repository. Solar is a PHP 5 framework for rapid application development.

On an administrative note, Clay Loveless has set up a Trac installation for us, so now you can report bugs and request enhancements without having to join the Solar-Talk mailing list. You can keep up with the Trac issues via the Solar-Trac mailing list if you like.

You can read the change notes for a full run-down, but there are a few changes in particular that I want to highlight.

Solar_Request

In previous releases, the Solar arch-class had a series of methods to read from the superglobals: Solar::post(), for example, would read a $_POST key and return it (or a default value of your choosing if the key did not exist). Those methods and their related support methods have been removed from the arch-class and placed in their own class, the new Solar_Request.

For now, Solar_Request acts as a singleton, since it is tied back to the super globals. Future releases will convert it to a true standalone object, and classes that need access to the request environment will receive a dependency injection of a Solar_Request object.

Whereas you would have used Solar::post() in earlier versions of Solar, you now create a Solar_Request object and use the $request->post() method. Solar_Request provides access to these superglobals after some minimal filtering (essentially just stripping magic quotes):

  • get() -- $_GET
  • post() -- $_POST
  • cookie() -- $_COOKIE
  • env() -- $_ENV
  • server() -- $_SERVER
  • files() -- $_FILES
  • http() -- ...

Wait a minute, there's no "HTTP" superglobal! One of the things that Solar_Request does for you is some basic cleaning and normalizing of the $_SERVER['HTTP_*'] values into their own storage array, so you can ask for the HTTP headers directly. For example, to get the value of the "X-Requested-With" header, ask for $request->http(‘X-Requested-With').

Solar_Request also lets you check how the current request was made:

  • isGet() is true if this was a GET request
  • isPost() is true if this was a POST request
  • isPut() is true if this was a PUT request
  • isDelete() is true if this was a DELETE request
  • isXml() is true if this was an XMLHTTP request (Ajax!)

Many thanks to Clay Loveless for getting Solar_Request up and running.

Solar_Session

Similar to Solar_Request, Solar_Session (here) provides an interface to the $_SESSION superglobal. It also takes over for the previous Solar_Flash class by providing read-once "flash" support via the $_SESSION values.

In Solar, we aggressively segment the global name space so that different classes don't step on each other's toes. Solar_Session makes this segmentation easier, by providing a config key to tell the particular instance what class name space it should be working within the $_SESSION array.

Progressive Ajax Enhancement

It's not much, but I've added the tiniest bit of Scriptaculous enhancement to the reference application that comes with Solar, Solar_App_Bookmarks. In the "edit" view, instead of using a simple class to indicate success or failure of a save, I've put in a pair of effect calls:

// output a form from the "formdata" processor
echo $this->form(array('id' => 'form-bookmark'))
      ->auto($this->formdata)
      // ...
      ->fetch();

// add an effect for success message lists
$this->jsScriptaculous()->effect->highlight(
    "#form-bookmark ul.success",
    array(
        'duration' => 3,
        'endcolor' => '#aaaaff',
        'restorecolor' => true,
    )
);

// add an effect for failure message lists
$this->jsScriptaculous()->effect->highlight(
    "#form-bookmark ul.failure",
    array(
        'duration' => 3,
        'endcolor' => '#ffaaaa',
        'restorecolor' => true,
    )
);

That's all it takes. Again, Clay Loveless has done wonderful work with his JavaScript helpers for Solar.


Zend PHP 5 Certified

I just got a note from Dhwani Vahia that I passed the Zend certification exam for PHP 5. Woohoo! :-)

Many thanks to the folks at PHP Architect for making the test available free of charge at php|works. That's twice now I've been able to take it on their dime (although the test in Cancun last May was a lot more fun, for obvious reasons ;-).


AJAX Is Interesting, But The Basics Are Better

I had the good luck to share a cab back to the airport with Wez Furlong after php|works. Wez had presented a new talk about how to work with email properly (jokingly subtitled "Not PDO" by everyone there), and was curious to know how it was received. I replied that I liked it, but struggled for a moment on "why" ... and then it hit me: the reason I liked it was that it was about a fundamental operation that still seems to trip developers up regularly.

Although I told Cal Evans when he interviewed me that the next technology I'm really interested in is AJAX (particularly its implementation in Protaculous), I'm not as excited by it as everyone else seems to be. I think it's neat, but good grief, if wide swaths of developers can't do input filtering properly, what good can adding AJAX do? Get the basics right first, then you can do progressive enhancement as you go.



Teaser for php|works

I've noted before that I'll be presenting at php|works in a couple of days; it's in the 4-5pm slot on Wednesday (13 Sep). The synopsis is public, but I wanted to give a little teaser about the presentation, in hopes of drumming up greater attendance. The talk is titled "Organizing Your PHP Projects", and the subtitle is "Project Planning in One Lesson."

Fans of Henry Hazlitt's "Economics in One Lesson" will recognize the kind of talk I have in store, although I don't have the time available to expound in detail the way Hazlitt does. The talk will consist of a one-sentence lesson for you to remember when deciding how to organize your PHP project, whether it's a library, an application, a CMS, or a framework. I'll then explain how to apply the One Lesson in your project, and the various follow-on effects the One Lesson will have on your project organization.

Also, I can guarantee that I will be the only presenter to provide examples and analogies using such sources as livestock, Jimmy Hoffa, and the Mir space station in a relevant and entertaining fashion. ;-)

Finally, as part of the research for this talk, I examined 40+ public PHP projects, and found a few distinct organizational patterns that I'll give in some detail. Among other things, you will learn exactly what the difference is between a framework and a CMS, and how they got to be that way. Here's one of the core slides on that part of the talk:

See you at php|works!

UPDATE: (2006-09-14) The slides from my talk are now available as a PDF file. (I'll create an audio version of the presentation after I get back from the conference.)