Category: CakePHP

Improved cache and sessions storage with CakePHP and Redis

CakePHP 2.3 + Redis 2.6.7

For the vast majority of applications out there the issue of cache and session store doesn’t really come into play until the application is large enough to where it needs to be scaled to multiple web servers.
At that point filesystem (default cache and session store) becomes inadequate for a couple of reasons:

  1. You need a central location for sessions because maintaining user sessions on individual servers will be extremely painful. You’d have to persist the user to the same web node.
  2. If you clear/invalidate the cache, it has to be done on all servers consistently or your application will be in a very unpredictable state.
  3. Writing to filesystem is relatively slow.

Often a simple enough solution is to use the database to keep track of sessions across all web servers. This option is nicely integrated into cake, and doesn’t require much in terms of configuration (other than changing the default value from “php” to “database” in core.php).

Today we’ll look at setting up Redis to be used as a cache and sessions storage. Although we are not going leverage some of the advanced features, the benefits of having a very fast, memory-based store with out-of-the-box persistence are great enough for any application, whether you run off of one server or twenty five.

Now that we know “why”… let’s see “how”…

First, we are going to install Redis server. For those on Ubuntu the installation process is honestly a little too easy:

apt-get install redis-server

Done.

Working with default config settings is recommended for a basic installation. (Cake will actually expect certain Redis defaults).

Now that we have our server installed and running, we need to tie PHP and Redis by using any of the available libraries.
The one I like is:
https://github.com/nicolasff/phpredis

The installation is rather simple also.
Clone the repository and run the three commands provided at the link above.

Finally, let’s configure our application to use Redis. All we have to do is modify core.php

First, change the value of $engine to Redis

$engine = 'Redis';

Next, let’s create a separate cache config for our sessions.

Cache::config('session', array(
    'engine' => $engine,
    'prefix' => $prefix . 'cake_session_',
    'duration' => $duration
));

Now we’ll tell cake to use the cache store for sessions as well. The “handler” key ties everything together.
(Ability to use cache store for sessions is an awesome addition to CakePHP and just as well you could use APC, memcached, or some other solution).

Configure::write('Session', array(
        'defaults' => 'cache',
        'timeout' => 100,
        'start' => true,
        'checkAgent' => false,
        'handler' => array(
            'config' => 'session'
        )
    ));

With all of this in place, we can now test our application to make sure everything is working as expected:

  1. CakePHP core Redis Engine tests should be passing
  2. No cache or sessions files are written to local file system
  3. We can run redis-cli and then “KEYS cake*” to see all cake-related keys in the data store. There should be cake_session_*, cake_model_*, cake_core_* keys listed. (This presumes you’ve tried to use the app by clicking around and possibly logging-in)

Million dollar site in CakePHP …

Is that possible? Yes, and it has been done.

But…

— CakePHP is slow. I’ve seen bench-marks on “Hello world” and cake falls behind other frameworks.
— What’s your caching strategy?
— [crickets] …

  1. Cache content
  2. CakePHP comes with a built-in view caching mechanism. Granted there is always a question of real-time data vs. performance, but I’ve yet to come across a project where at least some content could not be cached. Even a five or ten minute cache can be quite helpful if you’ve got millions of hits a day on your site. I’ve described some strategies in a post a while back, so give it a read, if you are after the details of implementation.
    If you need something faster, consider adding a front-end cache solution, like varnish or reverse-proxy nginx. (If you have a choice consider replacing apache, which requires a lot of tweaking with nginx as your web server, which is quite fast out of the box).

  3. Cache DB queries
  4. Optimizing queries is important. You should always use containable and limit the fields being returned by each query. This optimization will only take you so far, however… and I would argue that before you go head-first into nitty-gritty details of each query, you should invest into setting up memcached (nicely supported by cake) to alleviate some load on your DB.

  5. Index your DB fields
  6. Do you have some JOIN’s? Do you perform some searches on certain fields? (i.e. “username”). Then you’d better remember to index any fields in your DB that are used in the JOIN or being searched on. There are a few ways to properly use indexes, but one thing is for sure — without them, your DB and your app performance is going to suffer greatly.

  7. De-normalize your DB structure, better yet use an appropriate DB system
  8. Speaking of JOIN’s, no matter how you slice it… they are costly for the performance. Sometimes it helps to de-normalize your DB structure in order to avoid such expensive operations. An example would be a users table and user_profiles table. I love to keep minimal information in the users table, but if I find that I keep JOIN’ing user_profiles to get additional information, it is probably time to consider de-normalizing data and move whatever piece of info I need into the users table to avoid an extra JOIN. (Counter cache is another prime example of this).
    That being said, if you find yourself de-normalizing your DB quite heavily, perhaps it is time to consider an alternative to RDBMS. Of course, I am going to suggest MongoDB. Where applicable, it is quite alright to use a mixture of DB systems… always use the right tool for the right job (simple, but powerful statement).

  9. Create read/write DB replicas
  10. Do evaluate your read and write queries. Do you have a ton of admin features, which require complex find() operations? Offload them to a read replica, so that your users (or front-end) write queries do not get in the way. Increasing performance through replication is a nice trick, but you should be cautious not to offload mission-critical data to a replica, because the data might be a little behind as compared to your master server.

  11. Offload heavy tasks to a background process
  12. I do hear this pretty often… “I need to export data to an Excel, and it is taking forever”. This is a perfect example of a job that can be offloaded to a background operation. There is simply no need for a user to sit in-front of the screen and wait for X minutes for the Excel file to build and download. When a user requests an Excel report, add this task to your job queue manger and notify the user when the job is complete. (Have you looked at gearman?).
    — Well, I need to have this real-time.
    — Sorry, but this is not going to happen. Having a user sit and wait is already not “real-time”, IMO. Not to mention the unnecessary stress this kind of operation will put on your DB. (This kind of requirement is a perfect time to consider replication as well).

  13. Use AJAX, when needed
  14. The whole point of AJAX was to minimize the number of heavy requests to the server. Imagine you have an e-commerce site, where you have a page of most popular t-shirts (purchased by the users in the last week) cached… for a week. Makes sense, you only rebuild this list once a week, based on the updated data in the DB. For a week, this popular page on your site is served statically, just like good ol’ HTML.
    But there is a gotcha… you have shopping cart info also as part of this page. Well, AJAX to the rescue. While the rest of the page is cached, one little div, which has the shopping cart summary is easily updated by AJAX.
    Granted this is a very simple example, but think about how well this applies to other situations where you need to mix and match static and dynamic content.

A few other things worth looking into: APC, hiphop-php, Yahoo! performance rules.

Overall, I suggest not to get too hung up on trying to squeeze milliseconds out of your PHP code, unless you’ve exhausted all other resources. Spend your time on proper architecture and caching strategies. Avoid premature optimization and do not trust “hello world” benchmarks when measuring something so complex as an entire framework.
Oh, and before you invest hours into setting up various caching and optimization tools, do not forget to run a load and stress tests to help you identify early bottle-necks upfront.

Would love to hear your experience with optimization, caching and improving performance.

CakePHP + MongoDB, Next Steps

CakePHP 2.2 / MongoDB 2.0.4

Let’s continue building on top of what we’ve done previously.

I’d say that the main consideration when it comes to building out your MongoDB is the choice between embedding everything into a single collection vs linking (or manual referencing) to another collection.

What does it mean?

If we take our blog example, we saw how easy it was to build a tiny app, which allows to add posts. No need to create any tables (collections) or define any sort of schema. Just get the data and insert.
Now we are ready to move forward and our application would probably require an ability to add comments. Because MongoDB is schema-less, first thought would be to store the Post and Comments into a single document. After all, this is the beauty of NoSQL, we don’t need any JOIN’s, we don’t have to worry about belongsTo or hasMany. Again, a simple head-on approach would be: get the data and insert it into the collection of documents (i.e. Posts and Comments are stored together, aka “embedding”).

This is all fine and dandy, and will likely work quite well for our simple example.
However, this post would be useless if we didn’t consider some other issues. For instance, it would become a bit more difficult (actually it can become very difficult in a more complex application) to single out a specific comment. Perhaps you need to edit one comment out of a hundred or so. Having Comments embedded with the Posts, will require a bit of hacking around. (I strongly urge you to read this wonderful post, which details some these considerations: http://seanhess.github.com/2012/02/01/mongodb_relational.html).

So, for the sake of the example (at least), we’ll separate our collections into something more RDBMS-like and create Post and Comment separately (aka “linking” or “manual referencing”).
As you probably suspect we’ll use the same approach as we would in a traditional SQL DB, and create a reference field in the Comments collection… post_id anyone?

Side note: the referencing that we are doing here is called “manual referencing” in MongoDB terms. There is also an option to use DBRefs, which a slightly more advanced approach. In most cases, and such as the one we have here, “manual referencing” is the recommended way to structure data. Also, at the time of writing PHP’s MongoDB driver does not yet support DBRefs, it is however in the works.

With all this mind let’s take a look at our CakePHP view, app/View/Posts/view.ctp:

<?php if (!empty($post)) : ?>
  <h3>
    <?php echo h($post['Post']['title']); ?>
  </h3>
  <p>
    <?php echo h($post['Post']['body']); ?>
  </p>

  <?php if (!empty($comments)) : ?>
    <?php foreach($comments as $comment) : ?>
      <p>
          <?php echo h($comment['Comment']['user']); ?>
      </p>
      <p>
          <?php echo h($comment['Comment']['body']); ?>
      </p>
    <?php endforeach; ?>
  <?php endif; ?>

  <?php
    echo $this->Form->create(array(
      'url' => array(
        'controller' => 'comments',
        'action' => 'add'
      )
    ));
    echo $this->Form->inputs(array(
        'legend' => 'Add some comments:',
        'Comment.user',
        'Comment.body'
        )
    );
    echo $this->Form->input('Comment.post_id', array(
        'type' => 'hidden',
        'value' => $post['Post']['_id']
    ));
    echo $this->Form->end('Add comment');
  ?>
<?php endif; ?>

Nothing different here from our typical CakePHP view. The page will display a post, some comments (if available) and a form to add more comments.
As evident from the form, we should create a Comments Controller with an add() action. To allow the saving of the comments.

Let’s do so right now:

<?php
class CommentsController extends AppController {
       
public function add() {
        if ($this->request->is('post')) {
            if ($this->Comment->save($this->request->data)) {
                $this->Session->setFlash('Your comment has been saved.');
                return $this->redirect(array(
                    'controller' => 'posts',
                    'action' => 'view',
                    $this->request->data['Comment']['post_id']
                ));
            } else {
                $this->Session->setFlash('Unable to add your post.');
            }
        }
    }
}

This is all that we would need in order to start saving Comments and their link (via post_id) to the actual Post. Again, since we are dealing with a schema-less DB there is no need to worry about creating anything on the DB layer. Our Comment model is also not required, because we don’t have any validation logic or custom methods, so an App Model instance, which is created by cake for us is sufficient for this basic operation.

Now, that the comments are saved we should also display them on the “view post” page. If you take a look at the view above, you’ll see that we’ve already setup the logic to display the comments and we’d expect a $comments data array to do so.

Here’s our view() method in the relevant Posts Controller:

public function view($id = null) {
      $this->Post->id = $id;
      $this->set('post', $this->Post->read());
      $this->set('comments', ClassRegistry::init('Comment')->find('all', array(
          'conditions' => array(
            'post_id' => $id
          )
      )));
    }

Whoa… what happened to find(‘all’)? Why did I just issue two separate queries, and had to instantiate a Comment model like that?
Before you despair, let’s summarize a few things:

  1. We did not setup any models, so our relationship between Post and Comment is unknown.
  2. Currently the MongoDB driver does not support relationships as you would expect from a typical RDBMS. This is not to discourage you form the beauty of using an ORM. On the contrary, MongoDB is not meant to have deep-linking and complex relationships between collections. Remember, MongoDB is all about de-normalization. (That being said, there’s no reason why cake can’t support a basic one-level association, the driver is simply not there yet).
  3. Considering the above, this is not to say that we don’t need models at all. I always mention that all business logic should be tucked away in the model as much as possible. So this is not an excuse to make our controllers fat. The example is simple enough, where extra code would be wasteful.

I’m going to wrap things up at this point, as this should already give you some interesting ideas and food for thought. You’ll notice that I’ve embedded the Poster’s name with the Comment. Does that mean that I would take the approach of storing all user data with the Comments? No, not really. Here it is just a display name, which can be derived from the form or a session if we had logged-in users in the system.
Further considerations and examples of this will be a good topic for another day.

CakePHP + MongoDB Introduction

CakePHP 2.2/MongoDB 2.0.4

We all know that CakePHP is awesome, and I’m sure you’ve heard that MongoDB is pretty awesome as well.
So how do we make these awesome technologies play well together?

In an awesomely easy way :)

Let’s get things rolling by setting up MongoDB.
(Instructions for Ubuntu)

sudo pecl install mongo

Add the following line to your php.ini file (on my system in: /etc/php5/apache2/php.ini)

extension=mongo.so

This sets up PHP support for MongoDB.

Let’s install the actual DB, which is easily available through apt-get

sudo apt-get install mongodb mongodb-server

Restart things:

sudo /etc/init.d/apache2 restart

… and double-check that mongo is up and running.

mongo --version

You should see something like: “MongoDB shell version: 2.0.4″

Now, we’ll grab the MongoDB driver for CakePHP, courtesy of Yasushi Ichikawa (ichikaway).

First, cd into your app/Plugin directory.
Then:

sudo git clone git://github.com/ichikaway/cakephp-mongodb.git Mongodb

Once the cloning is complete you should cd into Mongodb directory and:

git checkout cake2.0

Now you have all the components installed and running, and it’s time to try out a simple app.
There is certain beauty in MongoDB, because it doesn’t require you to have any schema. Just write some code, and things will happen magically.

Indeed we’ll just edit our app/Config/database.php

public $default = array(
    'datasource' => 'Mongodb.MongodbSource',
    'database' => 'blog',
    'host' => 'localhost',
    'port' => 27017,
  );

Add to app/Config/bootstrap.php

CakePlugin::load('Mongodb');

Create a model:

<?php
class Post extends AppModel {
 
  public $validate = array(
    'title' => array(
        'rule' => 'notEmpty'
    ),
    'body' => array(
        'rule' => 'notEmpty'
    )
  ); 
}

Controller:

<?php
class PostsController extends AppController {
   
    public function index() {
        $this->set('posts', $this->Post->find('all'));
    }
     
    public function add() {
        if ($this->request->is('post')) {
            if ($this->Post->save($this->request->data)) {
                $this->Session->setFlash('Your post has been saved.');
                return $this->redirect(array('action' => 'index'));
            } else {
                $this->Session->setFlash('Unable to add your post.');
            }
        }
    }
}

And a couple of views…

app/View/index.ctp

<?php
  foreach ($posts as $post) {
    debug($post);
  }

app/View/add.ctp

<?php

echo $this->Form->create();
echo $this->Form->inputs(array(
    'title', 'body'
));
echo $this->Form->end('Add');

Congratulations! Your first CakePHP/MongoDB app is now ready to rock.

You can go ahead and try adding some posts, and see them appear (albeit only as debug output) on the index action.

Building “the blog tutorial”… the TDD way (part 2 – controller testing)

CakePHP 2.2

We have prepped our app with enough testing, to be ready to move on to write out the rest of the code…
There was a good reason why we have started our TDD with the model layer. By encapsulating the business logic into our models we were able to test our application quite well, without having to write any controller code.

Now, feeling a lot more confident, let’s go ahead and create app/Controller/PostsController.php:

<?php
class PostsController extends AppController {
   
    public function index() {
        $this->set('posts', $this->Post->getAllPosts());
    }
}

We already know that our getAllPosts() method is working well, because it’s been tested in the model. Therefore our controller test will simply re-enforce that knowledge and complete the cycle by making sure our expectation of variables set() for the view matches the return. One simple example of where this could be an issue is a heavy and complex Post model, which has a ton of methods.
It is not that far-fetched to write:

$this->set('posts', $this->Post->getEachPost());

Perhaps, we’ve inherited the app from another developer or simply weren’t sure what getEachPost() method does, or made the wrong presumption that this is the method we need to be using for the index action. Thus, creating a simple test case can secure out intent and expectation on the controller level.

app/Test/Case/Controller/PostsControllerTest.php:

<?php
class PostsControllerTest extends ControllerTestCase {
    public $fixtures = array('app.post');
       
    public function testIndex() {
      $this->testAction('/posts/index');
      $this->assertInternalType('array', $this->vars['posts']);
      $expected = array(
          array(
              'id' => 2,
              'title' => 'A title once again'
          )
      );
      $result = Hash::extract($this->vars['posts'], '{n}.Post[id=2]');
      $this->assertEquals($expected, $result);
     
    }

As you see we’ve used the same fixture as we did in part 1, while testing our Post Model. Here, we have leveraged the index() action of the Posts Controller and made sure that our record with ID = 2, matches our expected return from the Post Model and sets the correct output to be used in the view. You’ll notice the use of $this->vars, which is simply a CakePHP way of storing the view variables within the testing environment.
The two assertions we have tested for:
1. That we have some array of variables ready for display (i.e. our Posts).
2. That a record with ID = 2, is the Post.id and Post.title, which, as we know, ultimately comes from our Post Fixture.
Once again, we have now bridged our test to cover everything from the model layer to controller to the setting of the view variables.

Our view() action and test are not going to be much different, and since we are well familiar with what our output and expectations should be, let’s add the missing bits to move on with our application development.

First we’ll add the new action to our Posts Controller:

public function view($id = null) {
      $this->set('post', $this->Post->getSinglePost($id));
    }

And the test case:

public function testView() {
      $this->testAction('/posts/view/3');
      $this->assertInternalType('array', $this->vars['post']);
      $expected = array(
          'Post' => array(
            'id' => '3',
            'title' => 'Title strikes back',
            'body' => 'This is really exciting! Not.',
            'created' => '2012-07-04 10:43:23',
            'updated' => '2012-07-04 10:45:31'
          )
      );
      $this->assertEquals($expected, $this->vars['post']);
    }

By now you should be armed with enough knowledge to figure out this simple addition to our app on your own.

Now we can get into something a little more interesting and take a look at how we would use Mock Objects to test our add() action.
This is a nice show-off of the simplicity of such a powerful feature and how nicely it is integrated in CakePHP. Sometimes we don’t need or don’t want to have a full-blown app written just to test bits of logic (as the case might be with the simple add() method in our little app). Therefore cake will generate “fake” objects (i.e. Mock Objects) and leverage some, but not all, of the existing code to do our testing.

Let’s beef up the Posts Controller with the new add() action:

public function add() {
        if ($this->request->is('post')) {
            if ($this->Post->addPost($this->request->data)) {
                $this->Session->setFlash('Your post has been saved.');
                return $this->redirect(array('action' => 'index'));
            } else {
                $this->Session->setFlash('Unable to add your post.');
            }
        }
    }

… and let’s take a look at the test case:

public function testAddViaMock() {
      $postData = array(
          'Post' => array(
            'title' => 'New Post Title',
            'body' => 'TDD FTW!'
          )
      );
           
      $this->testAction('/posts/add', array(
          'data' => $postData,
          'method' => 'post'
          )
      );
     
      $this->assertContains('http://app/posts', $this->headers['Location']);
      $this->assertEquals(4, $this->controller->Post->find('count'));    
    }
   
public function testAddViaMockWithBadData() {
      $postBadData = array(
          'Post' => array(
            'title' => '',
            'body' => 'TDD FTW!'
          )
      );
     
      $this->testAction('/posts/add', array(
          'data' => $postBadData,
          'method' => 'post'
          )
      );
           
      $this->assertTrue(!empty($this->controller->Post->validationErrors));
      $this->assertContains('This field cannot be left blank', $this->controller->Post->validationErrors['title']);
      $this->assertEquals(3, $this->controller->Post->find('count'));
      $this->assertTrue(empty($this->headers));
    }

As you see we’ve actually added two methods to test some assertions. One comes with “good” data and one with “bad” data (empty title). This helps us to keep things more granular and allows to test our if/else conditions all the way to saving of the data and model validation.

Behind the scenes cake will Mock a controller object for us, meaning it created a “fake” Posts Controller. Although in reality the framework still leverages the existing code, yet only minimal parts are required for testing. There is an option to fine-tune the creation of such Mock Objects by using the generate() method, but in this case we can get away by relying on the already created model, fixture and bits of the controller.
I mentioned before that models are easier to test and that it is always a good idea to keep the logic in the model, hopefully you can start to see why. Had we placed all the logic in the controller the testing would become a lot more complicated and at the same time not nearly as precise as what we have now.
It does take a little a practice to write good test cases. However, by employing the TDD way of thinking we are naturally pushed towards making better decisions about logic separation and thereby necessity forces us to improve the overall architecture.

I will let you browse the code and take a look at the assertions, there really isn’t anything new there, that we have not covered yet. Just to summarize:
1. We test user input with good data, and make sure our if/else and redirection works as expected. Hint: $this->assertContains(‘http://app/posts’, $this->headers['Location']);. (Yep, the post was saved, therefore we got redirected back to “index”).
2. We make sure our new post is indeed saved as expected: $this->assertEquals(4, $this->controller->Post->find(‘count’));
3. In the second method we ensure that validation will kick-in, if a user forgets to add a post title.
4. And double-check that no data will be stored in this case.

This post is already getting rather long, so I will not bore you anymore by showing the edit() action and the test case for it. In reality it would be nearly identical to the add() action, and since we’ve seen these methods tested on the model layer it shouldn’t take you long to build up a test case on your own.

Building “the blog tutorial”… the TDD way (part 1 – model testing)

CakePHP 2.2

I personally consider Test Driven Development (TDD) as a “must do”, rather than a “nice to have”. Unfortunately as developers we often suffer from tight deadlines, tons of requirements and deliverables and sometimes a simple lack of understanding as to why TDD is so important.
Granted, TDD requires certain time and effort to be spent before “real” development begins (I am willing to argue this point), therefore unit testing becomes an afterthought. This is rather sad, because in the long run it will save you time, frustration and money. No longer will you have litter your application code with debug statements or try to trace logic of an application because something unforeseen and inexplicable is suddenly happening.

PHPUnit manual (a de facto standard in PHP unit testing and the testing framework leveraged by CakePHP) puts it quite nicely:

The sooner you test for a mistake the greater your chance of finding it and the less it will cost to find and fix. This explains why leaving testing until just before releasing software is so problematic. Most errors do not get caught at all, and the cost of fixing the ones you do catch is so high that you have to perform triage with the errors because you just cannot afford to fix them all.

Let’s take our favorite blog tutorial and approach it the TDD way.
I will presume that you are familiar with CakePHP and have read about testing within CakePHP environment already, and have setup your app to be ready for testing (i.e. installed PHPUnit, setup test DB… and possibly installed Xdebug. For details please refer to the CakePHP manual in the link above).

If we follow the blog tutorial, you’ll see that first we will be creating a Post Model. Since we are doing things via TDD, this sort of translates into the idea that we’ll need a Post Fixture. You could work with the live data instead, but since we are just getting started fixtures is a straight-forward and solid approach to unit testing.

So, app/Test/Fixture/PostFixture.php:

<?php
class PostFixture extends CakeTestFixture {
      public $fields = array(
          'id' => array('type' => 'integer', 'key' => 'primary'),
          'title' => array('type' => 'string', 'length' => 255, 'null' => false),
          'body' => 'text',
          'created' => 'datetime',
          'updated' => 'datetime'
      );
      public $records = array(
          array('id' => 1, 'title' => 'The title', 'body' => 'This is the post body.', 'created' => '2012-07-04 10:39:23', 'updated' => '2012-07-04 10:41:31'),
          array('id' => 2, 'title' => 'A title once again', 'body' => 'And the post body follows.', 'created' => '2012-07-04 10:41:23', 'updated' => '2012-07-04 10:43:31'),
          array('id' => 3, 'title' => 'Title strikes back', 'body' => 'This is really exciting! Not.', 'created' => '2012-07-04 10:43:23', 'updated' => '2012-07-04 10:45:31')
      );
 }

We’ll keep things simple and create the fixture with some test records, just like the blog tutorial suggests.
If you’ve read and tried the blog tutorial already, you’ll see that all of the logic is really contained in the controller. Additionally all the heavy lifting is delegated to the core methods such as save() and find().
It would be silly to test these core methods (as cake developers have already done a great job at that), therefore I am going to create wrapper methods in the model, so that we have something to test. First, because models are generally easier to test than controllers, and secondly because even in such a simple app we can yet again benefit from the golden rule of MVC “fat models, skinny controllers”.
Let us modify the code just a little to see what I mean.
Instead of writing out the controller just yet, I am going to create a method in our Post Model to get Post.id and Post.title for our soon-to-be-created index view.
(Since we’ve read the blog tutorial, we are aware of the expectations and can therefore approach our application with a TDD mindset).

app/Model/Post.php

<?php
class Post extends AppModel {

  public function getAllPosts() {
    return $this->find('all', array(
        'fields' => array('id', 'title')
    ));
  }
}

Our model now has one method, which I can test to be sure that my expectation of how it should behave matches the reality.
Thus, comes our first test app/Test/Case/Model/PostTest.php:

<?php
App::uses('Post', 'Model');

class PostTest extends CakeTestCase {
    public $fixtures = array('app.post');

    public function setUp() {
        parent::setUp();
        $this->Post = ClassRegistry::init('Post');
    }

    public function testGetAllPosts() {
        $result = $this->Post->getAllPosts();
        $expected = array(
            array('Post' => array('id' => 1, 'title' => 'The title')),
            array('Post' => array('id' => 2, 'title' => 'A title once again')),
            array('Post' => array('id' => 3, 'title' => 'Title strikes back'))
        );

        $this->assertEquals($expected, $result);
    }
}

If we execute our test, the following output should be shown: “1/1 test methods complete: 1 passes, 0 fails, 1 assertions and 0 exceptions.” (If Xdebug is installed the coverage should be at 100%, w00t!).

This is all good news… but what have we accomplished?

First of all, without having to populate our application table with any testing data we were able to make sure that our getAllPosts(); method is working as expected, thanks to our Post Fixture. Secondly, I am quite confident now that once I start writing the controller code I will not run into any trouble.
Pfft… all this coding just to make sure that “The title” which I wrote in the fixture matches my expectation of “The title” in test case. Well, yeah. Yet, imagine a slightly more complicated app. I might already have some data in the DB, and now by using TDD and fixtures adding a new method to the model will not require me to add fake data to the table, mess up my pretty code with debug statements and so on and so forth.
Just like tight coupling is generally bad for your system architecture, coupling testing and production code is just as bad (if not worse) and wasteful. Also, this sets us up for a solid foundation and proper mindset for further growth and easy development.

Let us move forward, now that we have prepared our model/method to build an index() action.
What comes next? Eventually, we would like to build a view() action, which is going to display a single post based on a given $id. With our TDD mindset, let us extend our Post Model and Post Test.

First we’ll add a new method to our Post Model

public function getSinglePost($id = null) {
    return $this->find('first', array(
        'conditions' => array('id' => $id)
    ));
  }

If you’ve been around CakePHP for a little while, chances are this method is going to work perfectly well. Although after years of development I can definitely misspell “conditinos” (ooops!). I am not taking any chances, not when my work comes down to 0 and 1 eventually, and therefore I will be sure to write a test case for it. If we were to analyze our code coverage at this point we only have tests for 62.5% of the application… bad developer.

Let’s add a test for the new method:

public function testGetSinglePost() {
        $result = $this->Post->getSinglePost(2);               
        $expected = array(
            'Post' => array(
                'id' => 2,
                'title' => 'A title once again',
                'body' => 'And the post body follows.',
                'created' => '2012-07-04 10:41:23',
                'updated' => '2012-07-04 10:43:31'
            )
        );
        $this->assertEquals($expected, $result);

Once again, if we were to run to the test the output should show that our app is at 100% coverage and that our method is indeed returning the record where ID = 2. For the sake of discussion try to write “condition” instead of “conditions” in our Post Model. The test immediately fails.
Yet, without unit testing I could’ve clicked on the link in the browser to see my post and possibly not even realize that cake returned the wrong record, in such case it would simply return the first record (clicked a link, saw a post, quickly move to the next step). Thus a perfect example of how a simple bug can slip by a developer, especially after hours of coding. Also, by testing in small increments I can quickly narrow down my attention to the specific method, and hopefully catch my mistake quicker. We all know that such issues become hard to spot after coding for hours and jumping all over the app.

Great, we are now ready to setup our add() method.

Let’s extend the Post Model even further:

public function addPost($postData) {
    return $this->save($postData);
  }

And let’s add a new test for this rather simple method:

public function testAddPost() {
      $postData = array(
          'title' => 'Test Post Title',
          'body' => 'We love TDD. Yeah!',
          'created' => '2012-07-04 10:43:23',
          'updated' => '2012-07-04 10:45:31'
      );
     
      $numRecordsBefore = $this->Post->find('count');
      $result = $this->Post->addPost($postData);
      $numRecordsAfter = $this->Post->find('count');
     
      $expected = array(
            'Post' => array(
                'id' => 4,
                'title' => 'Test Post Title',
                'body' => 'We love TDD. Yeah!',
                'created' => '2012-07-04 10:43:23',
                'updated' => '2012-07-04 10:45:31'
            )
        );
     
      $this->assertEquals(4, $numRecordsAfter);
      $this->assertTrue($numRecordsBefore != $numRecordsAfter);
      $this->assertEquals($expected, $result);
    }

Essentially we are just re-testing the save() method, but there are some interesting things happening here nonetheless. I know that my original record-set contained 3 rows, I would like to verify that by counting the number of rows before and after the save() operation. Because I am saving a single record from my $postData array, my expectation would be to now have four records.
So, as you see, in a single test I can make a number of “assertions” to really bullet-proof my expectations.
1. I expect the number of records to increase by one.
2. The number of records before and after the save() operation should not be the same.
3. My saved record will have a new ID, which is equal to 4 (since our primary key column is auto-incremented). Notice the difference between my data array and my expected array, which now has an ‘id’ column.
This demonstrates how a very simple method can be thoroughly tested using multiple assertions, so that I can peacefully sleep at night knowing that my code is working exactly as I expect it too.

Last, but not least, we have an edit() method to worry about. Let us proceed with creating an editPost() method and a relevant test case.

In app/Model/Post.php, we’ll add:

public function editPost($postData) {
    return $this->save($postData);
  }

And our test case:

public function testEditPost() {
      $this->Post->id = 3;
      $postData = array(
          'title' => 'Test Post Title. Updated.',
          'body' => 'We love TDD. Yeah! Yeah!',
          'created' => '2012-07-04 10:43:23',
          'updated' => '2012-07-04 10:49:51'
      );
      $recordBeforeEdit = $this->Post->read();
      $numRecordsBefore = $this->Post->find('count');
      $result = $this->Post->editPost($postData);
      $numRecordsAfter = $this->Post->find('count');
     
      $expected = array(
            'Post' => array(
                'id' => 3,
                'title' => 'Test Post Title. Updated.',
                'body' => 'We love TDD. Yeah! Yeah!',
                'created' => '2012-07-04 10:43:23',
                'updated' => '2012-07-04 10:49:51'
            )
        );
     
      $this->assertEquals($expected, $result);
      $this->assertTrue($numRecordsBefore == $numRecordsAfter);
           
      $recordCompare = array_diff($recordBeforeEdit['Post'], $result['Post']);
      $expectedArrayDiffResult = array(
        'title' => 'Title strikes back',
        'body' => 'This is really exciting! Not.',
        'updated' => '2012-07-04 10:45:31'
      );
     
      $this->assertEquals($expectedArrayDiffResult, $recordCompare);
    }

Whoa, again we have a rather simple method and rather thorough test with some important expectations. In this case we are modifying our record with ID = 3.
1. Since we are presuming an SQL UPDATE, our expected result should contain ‘id’ => 3 (as evident from the $expected array).
2. We need to be certain that, unlike in our addPost() operation, in this case the number of records remains the same.
3. Comparing the existing record to an updated one, we should have “title”, “body” and “updated” fields changed, while our “id” and “created” fields remain the same. We verify this by using PHP’s array_diff() and comparing our expectation to the actual output.

What about data validation?

Surely we can test that as well. Let’s add validation rules to our Post Model, just like suggested in the blog tutorial.

public $validate = array(
    'title' => array(
        'rule' => 'notEmpty'
    ),
    'body' => array(
        'rule' => 'notEmpty'
    )
  );

And a test case, to seal the deal:

public function testValidation() {
      $postData = array(
          'title' => '',
          'body' => 'Oh no, this post has and empty title!'
      );
      $result = $this->Post->addPost($postData);
      $invalidFields = $this->Post->invalidFields();
 
      $this->assertFalse($result);
      $this->assertContains('This field cannot be left blank', $invalidFields['title']);
     
      $postData = array(
          'title' => 'No body in the post? Impossible.',
          'body' => ''
      );
      $result = $this->Post->addPost($postData);
      $invalidFields = $this->Post->invalidFields();
     
      $this->assertFalse($result);
      $this->assertContains('This field cannot be left blank', $invalidFields['body']);
     
      $postData = array(
          'title' => 'Title...',
          'body' => '... and body.'
      );
      $result = $this->Post->addPost($postData);
      $invalidFields = $this->Post->invalidFields();
     
      $this->assertFalse(empty($result));
      $this->assertTrue(empty($invalidFields));
    }

First we are trying to save a post with an empty “title”, which should not be allowed. We prove that by asserting that our $result will be false. We will also verify that the proper validation error was triggered for the given field, “title”. We do exactly the same for the “body” of the post.
Of course, we should also make sure that given some “title” and some “body” the post will be saved, otherwise we could not be certain that our validation rules are not preventing a perfectly valid data to be saved at all. In this case we could compare $result to the actual expected array, but we have done so already while testing the addPost() method. Just a couple of simple false/true assertion will solidify our testing scenarios.
In theory, and this is something I would recommend for a more complex case, we could break up our testValidation() method into testEmptyTitle(), testEmptyBody(), testGoodTitleBody() methods.

In summary, our business logic of the application has been thoroughly tested, we can list multiple posts, we can view a single post, we can add and edit posts and we’ve made sure that our validation rules are working as expected.
Now I am content to commit my code and continue working on the application or pass it onto front-end engineers to build views or another developer to work on controllers. Talk about taking good care of your teammates and taking good care of your coding. Also, we didn’t even write a single line of code in the controllers or the views, but we can be certain that the logic of the application is quite solid. Do you see how TDD is playing so well with MVC paradigm of keeping our business logic in the Model?

In the next part we will continue building and putting finishing touches onto our awesome TDD app.

User Auth with CakePHP 2.1 – part 3

As promised in the previous part we’ll take a look at the admin section.

If you remember, we’ve setup our users so that when they create an account, they are inactive by default and cannot login into the app.

app/Controller/AppController.php

$this->Auth->authenticate = array(
            'all' => array (
                'scope' => array('User.is_active' => 1)
            ),
            'Form'
        );

Since our freshly created users have is_active = 0 we’ll need to create an admin page where one would be able to approve inactive users.
Let’s see how we’ll act in the admin role.

In part 1, we’ve setup prefix routing and our app is ready to accept admin logins.
This is our login() method again:

public function login() {
            if ($this->request->is('post')) {
                if ($this->Auth->login()) {
                    if ($this->Auth->user('is_admin')) {
                        return $this->redirect(array(
                            'controller' => 'users',
                            'action' => 'index',
                            'admin' => true
                        ));
                    } else {
                        return $this->redirect('/');
                    }
                } else {
                    $this->Session->setFlash(__('Username or password is incorrect'), 'default', array(), 'auth');
                }
            }
        }

And here’s a relevant view, just in case:

<?php
    echo $this->Form->create();
    echo $this->Form->inputs(
        array(
            'username',
            'password'
        )
    );
    echo $this->Form->end('Submit');
    echo $this->Html->link('Don\'t have an account? Register now!', array(
        'controller' => 'users',
        'action' => 'add'
    ));
?>

If we look at the redirect() method in the above login() it presumes that we have an admin_index() action.
Let’s create one:

public function admin_index() {
            $users = $this->paginate($this->User, array(
                'User.is_active' => 0
            ));
            $this->set(compact('users'));
        }

The goal is to display (paginate) inactive users, so that an admin can approve them… simple enough.
I’m not going to cover pagination setup here, as it is very simple, covered nicely in the manual and would be beyond what we need consider for now.

Anyways, let’s build a simple view to display our inactive users:

<?php if (isset($users) && !empty($users)) : ?>
   <?php foreach ($users as $user) : ?>
    <div>
        <?php echo $user['User']['username']; ?>
        <?php
            echo $this->Html->link('Activate', array(
                'controller' => 'users',
                'action' => 'user_activate',
                'admin' => true,
                 $user['User']['id']
            ));
        ?>
        <?php
            echo $this->Html->link('Edit', array(
                'controller' => 'users',
                'action' => 'user_edit',
                'admin' => true,
                 $user['User']['id']
            ));
        ?>
    </div>
   <?php endforeach; ?>
<?php endif; ?>

Next to each username we’ll show an “Activate” and “Edit” links.
Ultimately the “Activate” link we’ll be something like example.com/admin/users/user_activate/345. This should change the user status with ID = 345 from inactive to active.
Here’s the method to do so:

public function admin_user_activate($id = null) {
            if ($id) {
                $this->User->id = $id;
                if ($this->User->saveField('is_active', 1)) {
                    return $this->redirect(array(
                        'controller' => 'users',
                        'action' => 'index',
                        'admin' => true
                    ));
                }
               
            }
        }

All we are doing is updating is_active field to “1″ for a given user $id.
Now the user has been approved and we redirect the admin back to the index page.

Let’s recap:

  1. We’ve setup basic Auth to allow users to login and register in the system
  2. We’ve added a check so that only active users can login
  3. We’ve setup admin/prefix routing to allow for creation of admin-only resources
  4. We’ve added a check so that only admins can access the above resources
  5. And finally, we’ve added the ability for admins to activate the users

p.s. Here’s a simple chunk of code that creates a “Log in/Log out” link so that users can act accordingly. You’ll probably want to add this to your layout or a relevant element.

<?php
echo $this->Session->check('Auth.User')
?
$this->Html->link(
             'Log out',
              array(
                 'controller' => 'users',
                 'action' => 'logout',
                 'admin' => false
              ))
:
$this->Html->link(
              'Log in',
               array(
                  'controller' => 'users',
                  'action' => 'login'
               ));
?>

With a simple check for presence of the Auth.User key (this is where Auth stores information about logged-in users) we know if the user is logged-in or not into the system, and by using a quick ternary operator we display either a “Log out” or “Log in” links.

The end.

User Auth with CakePHP 2.1 – part 2

CakePHP 2.1

Now that we’ve completed our basic setup for Auth, let’s take a look at the User.php model…

class User extends AppModel {
   
    public $validate = array(
      'username' => array(
          array(
            'rule' => 'notEmpty',
            'message' => 'Username cannot be empty'
          ),
          array(
            'rule' => 'isUnique',
            'message' => 'This username is already taken'
          )          
      ),
      'password' => array(
          array(
            'rule' => 'notEmpty',
            'message' => 'Password cannot be empty'
          ),
          array(
            'rule' => array('minLength', 4),
            'message' => 'Must be at least 4 chars'
          ),
          array(
            'rule' => array('passCompare'),
            'message' => 'The passwords do not match'
          )
      )
    );
   
    public function passCompare() {
        return ($this->data[$this->alias]['password'] === $this->data[$this->alias]['password_confirm']);        
    }
   
    public function beforeSave() {
        $this->data['User']['password'] = AuthComponent::password($this->data['User']['password']);
        return true;
    }
}

Nothing terribly interesting… we’ve got our validation rules setup and just a few simple methods to handle the rest.
Let’s go in reverse a little. Keep in mind that the new Auth system in CakePHP doesn’t hash passwords by default. This is actually great news for old-timers, because the work-arounds and somewhat hacky solutions we had to do in 1.x (just to deal with password comparison, for example) are gone and are now replaced with a single line of code in your beforeSave() method.
(I hope you see which one). By using AuthComponent::password() we encrypt the user’s password with a default hashing algorithm.
Thus the password is safely hashed in the DB and we do so just prior to saving the record (remember to always return true;) in the beforeSave() or nothing will be… saved).

Speaking of password comparison, you’ll notice that in our validation rules we have a custom method passCompare(). Because I’m lazy and too much typing leads to headaches, a simple one liner will take care of our needs. If passwords match the method will return true, else it will return false and that’s all we really need to validate or invalidate the given field.

For now this covers our User.php model.

Let us take a look at some interesting things in the UsersController.php.

public function beforeFilter() {
    parent::beforeFilter();
    $this->Auth->allow(array(
       'add', 'account_created'
    ));
}

We will allow two methods in our UsersController.php to be accessible by non-authorized (not logged-in) users.
Of course, it would be silly not to allow users to register their account, thus we allow add() method… also we’ll have a simple action account_created(), which is not going to do much of anything for the time being.

Alright, let’s take a look at the relevant add() method:

public function add() {
            if ($this->request->is('post')) {
                if ($this->User->save($this->request->data)) {
                    $this->Session->setFlash(__('Account created. An admin will need to activate it.'), 'default', array(), 'auth');
                    return $this->redirect(array(
                        'controller' => 'users',
                        'action' => 'account_created'
                    ));
                }
            }
        }

All we do here is accept the data from a form, validate it (by using the save() method) and if all goes well, we redirect the user to the “account created” page.
Remember all that prefix/admin routing we’ve setup before?
Well, as you can see our user is going to live in the system, but remain inactive until an admin logs-in and activates her. Recall ‘scope’ => array(‘User.is_active’ => 1)… freshly created account will have is_active = 0, so without admin approval nobody is getting in.

Let’s take a quick look at the add.ctp view:

<?php
    echo $this->Form->create();
    echo $this->Form->inputs(
        array(
            'username',
            'password',
            'password_confirm' => array(
                'type' => 'password'
            )
        )
    );
    echo $this->Form->end('Submit');
?>

As you can see I really went all out here… three fields: username, password, and password_confirm. Now compare these fields to the validation rules in our User.php model and you should see how the whole thing is coming together. I kept the example purposely oversimplified to just show how the data will be POST’ed from the form to the controller’s add() method, validated by the User.php model (with password hashing) and thereafter saved to the DB.

To wrap things up for part 2 of this tutorial, let’s take a look at the login() and logout() methods (also in the UsersController.php).

public function login() {
            if ($this->request->is('post')) {
                if ($this->Auth->login()) {
                    if ($this->Auth->user('is_admin')) {
                        return $this->redirect(array(
                            'controller' => 'users',
                            'action' => 'index',
                            'admin' => true
                        ));
                    } else {
                        return $this->redirect('/');
                    }
                } else {
                    $this->Session->setFlash(__('Username or password is incorrect'), 'default', array(), 'auth');
                }
            }
        }

The interesting thing about this method is the check for admin. if ($this->Auth->user(‘is_admin’)) { … , so if the user is an admin we redirect them to example.com/admin/users/index (this is all part of prefix/admin routing). To tell cake, which “route” it should take we supply ‘admin’ => true. Of course, this presumes that we’ll have an admin_index() method in the UsersController.php. As you see, all other users simply get redirected to the “/” (root) of your website.

The logout() method is as simple as could be:

public function logout() {
            $this->redirect($this->Auth->logout());
        }

In the next part we will take a look at the admin side of things and some little things like a login/logout link, which you’d expect in any site that has user Auth.

User Auth with CakePHP 2.1 – part 1

CakePHP 2.1

The example application in the CakePHP book does a very good job of covering setup and implementation of a basic Auth system.
Let’s continue building on that and cover a few other things, by adding a couple of more features and looking at some of the other things in more detail.

Good to say that cake started separating authentication and authorization as of 2.x release. While these concepts always go hand-in-hand, I feel it created a bit of confusion for beginners, because while separate in theory they were not clearly separate in implementation.

As always, your main players when it comes to Auth is AppController.php and User.php.
(Of course, UsersController.php is not to be forgotten about…)

Let’s go ahead and start with app/Controllers/AppController.php:

class AppController extends Controller {
    public $components = array('Auth', 'Session');
   
    public function beforeFilter() {
        $this->Auth->authorize = array('Controller');
        $this->Auth->authenticate = array(
            'all' => array (
                'scope' => array('User.is_active' => 1)
            ),
            'Form'
        );
    }
   
    public function isAuthorized($user) {
        if (($this->params['prefix'] === 'admin') && ($user['is_admin'] != 1)) {
            return false;            
        }
        return true;
    }
}

First, as always we will include the necessary components. For the time being it’s just: public $components = array(‘Auth’, ‘Session’);.

Authorization is going to be controller-based. Meaning, we will tell CakePHP to authorize (let users access resources) based on the Controller actions.
(It’s nice that you can now specify such setting directly in the $components array, but I kept it in the beforeFilter() to show a slightly different approach).

Next, comes our Authentication setup. Unlike Authorization, which answers who is allowed to get to what, Authentication checks if the user is indeed who she claims to be (handle login/logout).

We’ll be using good ol’ login form, thus the ‘Form’ key in our setup.
Moving on, I’ve added a “scope” of User.is_active => 1… this presumes that in our users table we have a field called is_active and therefore only active users can access the application (everybody whose is_active status is equal to 0 is denied by default). More on this a little further.

isAuthorized() is a sweet little method, which helps us to fine-tune our permissions.
The implementation is truly up to your needs, but let’s see what we’ve got going on in this example…

public function isAuthorized($user) {
   if (($this->params['prefix'] === 'admin') && ($user['is_admin'] != 1)) {
      return false;            
   }
   return true;
}

The basic premise here is that an admin can access admin-related resources within the app. (We’ll get into prefix routing setup a little further down the line).
The presumption is that we have an is_admin field in our users table and if the user is not an admin $user['is_admin'] != 1 she cannot access any resource that has an “admin” prefix… again more on that a little later.
Otherwise, for all average Joe’s, we say return true; access whatever you want (once authorized) as long as it’s not an admin-related resource.

Alright then, let’s take a look at the prefix routing.
The setup couldn’t be simpler just un-comment the following in your app/Config/core.php.
Configure::write(‘Routing.prefixes’, array(‘admin’));

This concludes our basic setup. We’ll take a look at the User.php model as well UsersController.php in the next part, which is coming soon here.

p.s. For those who wish to play around with the setup, here’s all you’d need to get started with the users table.

[sql]
DROP TABLE IF EXISTS `users`;
CREATE TABLE `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`username` varchar(255) CHARACTER SET latin1 DEFAULT NULL,
`password` varchar(255) CHARACTER SET latin1 DEFAULT NULL,
`is_admin` tinyint(1) DEFAULT ’0′,
`is_active` tinyint(1) DEFAULT ’0′,
`created` datetime DEFAULT NULL,
`modified` datetime DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
[/sql]

“Error: Database connection “Mysql” is missing, or could not be created”. CakePHP 2.0 — cake bake trouble

CakePHP 2.0

Nothing like coming back from a hiatus with a WTF? post.
My favorite “cake bake” started failing with the message in the subject; as well it seems that I am not the only one who had suffered from the problem… (just give it a quick check on google).

After a little research here’s a summary of issues and steps to check:

The problem is caused the the fact that the PDO drivers are not available when you run php5 in cli mode (i.e. “cake bake”).

  1. First thing you’d want to check is which php.ini file is being used by your php (in cli mode!).
  2. In your terminal (or command prompt), run: php –ini. This will show the location of the currently used files. In many cases it is possible that your system php is not the same as your web server php, which you might have installed with something like XAMPP. If this is the case you need to change your path to use the correct install of php, which should have the necessary drivers available.
  3. Next step would be to check your php.ini (which is sometimes broken into a few config files). Check the PDO settings to make sure you have the correct extension enabled (might be as simple as un-commenting a line). For example, I am using MySQL therefore I needed to have MySQL PDO extension available (when running in cli!). It was working perfectly well “on the web”.
  4. Turns out I needed to run: sudo apt-get install php5-mysql

That was unexpected a little since CakePHP had no connection problems to MySQL, but there you go. This should cover all possible cases.
If you find other solutions not outlined here, please share.