CakePHP and jQuery auto-complete revisited

CakePHP 2.3
jQuery 1.10.2
jQuery UI 1.10.3

I’ve realized that my old post about jQuery auto-complete and cake is still pretty popular, but hopelessly outdated.

Therefore, I figured it would be a good time to revisit that old post and give it an update.
We’ve come so far!

For this tutorial we will create a single auto-complete field using jQuery and jQuery UI.
Although to show off some features of CakePHP we will also create a model a controller a view and a JSON response (more on that later).

The goal is simple, we’ll have a field where we’ll type some name of a car maker. If at least one character was entered, we’ll show suggestions using jQuery UI’s auto-complete widget.

First we’ll start with the schema and some data.

Let’s create our cars table.

CREATE  TABLE `test`.`cars` (
  `id` INT NOT NULL AUTO_INCREMENT ,
  `name` VARCHAR(45) NULL ,
  `created` VARCHAR(45) NULL ,
  `modified` VARCHAR(45) NULL ,
  PRIMARY KEY (`id`) )
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8
COLLATE = utf8_unicode_ci;

Now let’s populate it with some popular brands:

INSERT INTO `cars`
(name, created, modified)
VALUES
( 'Aston Martin', now(), now() ),
( 'Acura', now(), now() ),
( 'Audi', now(), now() ),
( 'Bentley', now(), now() ),
( 'Bmw', now(), now()),
( 'Bugatti', now(), now() ),
( 'Buick', now(), now() ),
( 'Cadillac', now(), now() ),
( 'Chevrolet', now(), now() ),
( 'Chrysler', now(), now() ),
( 'Dodge', now(), now() ),
( 'Ferrari', now(), now() ),
( 'Ford', now(), now() ),
( 'Gmc', now(), now()),
( 'Honda', now(), now() ),
( 'Hyundai', now(), now() ),
( 'Infiniti', now(), now() ),
( 'Jaguar', now(), now() ),
( 'Jeep', now(), now() ),
( 'Lamborghini', now(), now() ),
( 'Lexus', now(), now() ),
( 'Lincoln', now(), now() ),
( 'Maserati', now(), now() ),
( 'Mazda', now(), now() ),
( 'Mercedes-Benz', now(), now() ),
( 'Mitsubishi', now(), now() ),
( 'Tesla', now(), now() ),
( 'Nissan', now(), now() ),
( 'Porsche', now(), now() ),
( 'Rolls Royce', now(), now() ),
( 'Subaru', now(), now() ),
( 'Tesla', now(), now() ),
( 'Toyota', now(), now() ),
( 'Volkswagen', now(), now() ),
( 'Volvo', now(), now() )

Let’s go ahead and create a new layout for this application. It will be pretty simple, let’s do something like this (this will be a new file in View/Layout/basic.ctp):

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Sample App</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="description" content="">
    <meta name="author" content="">
    <?php
      echo $this->Html->css('https://code.jquery.com/ui/1.10.3/themes/smoothness/jquery-ui.css');
    ?>
  </head>

  <body>
  <?php echo $content_for_layout; ?>

  <!-- our scripts will be here -->
  <?php echo $scripts_for_layout; ?>
  </body>
</html>

To give some style to the auto-complete field and the “suggest” drop-down, we’ll add the CSS file form jQuery’s built-in themes.

Next we’ll create a simple controller in Controllers/CarController.php:

<?php
  class CarsController extends AppController {

    public $layout = 'basic';

    public function index() {

    }
  }

The only thing we do differently from our standard controller setup, is specifying the layout… which matches the file name above (minus the .ctp part).
We are leaving the index() action empty for now.

And finally let’s take a look at the view in View/Cars/index.ctp:

<?php
  //let's load jquery libs from google
  $this->Html->script('https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js', array('inline' => false));
  $this->Html->script('https://ajax.googleapis.com/ajax/libs/jqueryui/1.10.3/jquery-ui.min.js', array('inline' => false));

  //load file for this view to work on 'autocomplete' field
  $this->Html->script('View/Cars/index', array('inline' => false));

  //form with autocomplete class field
  echo $this->Form->create();
  echo $this->Form->input('name', array('class' => 'ui-autocomplete',
               'id' => 'autocomplete'));
  echo $this->Form->end();

First, we load our jQuery libs from Google. Next, notice $this->Html->script(‘View/Cars/index’, array(‘inline’ => false)); this tells CakePHP that we need to load a JavaScript file from our webroot/js/View/Cars/index.js. I recommend keeping your .js files in a similar directory structure as your .ctp files.

Because we have array(‘inline’ => false) as a second argument, our script will be included in place of the $scripts_for_layout.

This pretty much completes our CakePHP setup. We now need some code to retrieve data from our DB and some JavaScript code to act on our “#autocomplete” field.
As you’ve probably guessed, this JS code will be located in webroot/js/View/Cars/index.js:

(function($) {
  $('#autocomplete').autocomplete({
        source: "/cars/index.json"
  });
})(jQuery);

That’s it… One thing to note here is the path “/cars/index.json”. By adding the .json extension to our request URL we’ll utilize cake’s built-in JSON View and format the response as JSON (or JSONP).

Let’s take a look at that now. We will need to beef up our Controller just a little:

<?php
  class CarsController extends AppController {

    public $layout = 'basic';

    public $components = array('RequestHandler');

    public function index() {
      if ($this->request->is('ajax')) {
        $term = $this->request->query('term');
        $carNames = $this->Car->getCarNames($term);
        $this->set(compact('carNames'));
        $this->set('_serialize', 'carNames');
      }
    }
  }

One thing you’ll notice is that we’ve added a RequsetHandler component. This is the magic in CakePHP that will properly handle our request from jQuery and allow us to set our response as a JSON object. You can find out more details about how ‘_serialize’ and RequestHandler work by reading up in the manual.

It is important to note that in your routes file you’ll need to enable the parsing of extensions. (i.e. index.json).
Simply edit app/Config/routes.php and add the following line to the file:

Router::parseExtensions();

Next you see that I am getting the list of model names from our Car model in the method called getCarNames().
This is because I’m trying to follow the golden rule of MVC: “fat models, skinny controllers”.
Although it’s easy to leave all the car-name-finding logic in the controller (and not have to create a model at all!), we’ll presume good architecture here and create a model to handle our data finding needs.

Here we go (app/Model/Car.php):

<?php
  class Car extends AppModel {

    public function getCarNames ($term = null) {
      if(!empty($term)) {
        $cars = $this->find('list', array(
          'conditions' => array(
            'name LIKE' => trim($term) . '%'
          )
        ));
        return $cars;
      }
      return false;
    }
  }

I use a standard find(‘list’) method of CakePHP to get car names from our table above. The data is returned in an array formatted in a way so that becomes very easy to return as a JSON object back to our jQuery. You can see that in the controller above.
First we set a variable for the view (as you’d do for any view) and then you “serialize” it to become a JSON object.

(By creating a Car model cake automatically associated it with “cars” table. Even if I didn’t actually crate a Car model file, cake would still be able to execute basic model methods as all of our methods extend the built-in core Model. This topic is a bit more advanced and you can find out more about by studying he API or checking up on our friendly IRC channel).

In conclusion, we have everything we need to have a fully functional auto-complete using CakePHP and jQuery/jQuery UI.
If you were to type-in “f” in the input field, you’d get a list with “Ferrari” and “Ford”.

Easy CakePHP logging to FireBug with DebugKit and FirePHP

CakePHP 2.x

This is espeically useful when you are working with ajax or just in general you don’t want to dump the data to the screen, which is often hard to read.

Using FireBug and FirePHP and CakePHP is pretty nicely detailed here.

Presuming all of that is working well, let’s see how to implement something nice and simple for our app.

First, let’s add a custom function to app/Config/bootstrap.php

/**
 * https://cakephp.lighthouseapp.com/projects/42880/docs-firecake
 *
 * @param  mixed $data     data to log
 * @param  string $logLevel log, info, warn, error
 * @param  string $label    give your data some label
 *
 */

function fclog($data, $logLevel = 'log', $label = null) {
    if (Configure::read() > 0) {
        App::uses('FireCake', 'DebugKit.Lib');
        FireCake::$logLevel($data, $label);
    }
}

Now we can use it anywhere in our app like so:

$myData = $this->SomeModel->find('all');
fclog($myData, 'error', 'Oh no we have a lot of data!');

The second argument (in this case “error”) has to correspond to one of the logging methods (log, info, warn, error).

If all goes well, You should see a nicely formatted array of data in your firebug console.

Object mockery and Cake

A few times we already had gone over unit testing… and why it is a good thing to do.

If you had been interested in this topic for a while, then I’m sure you’ve come across the concept of “Mock Objects”.
But really, what are they? And more importantly how do we use them?

So, to get started let’s look at the “official” mock object definition.

http://en.wikipedia.org/wiki/Mock_object

In object-oriented programming, mock objects are simulated objects that mimic the behavior of real objects in controlled ways. A programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts.

Actually, this is a pretty good analogy. It does make sense and lets us work out a real example.

In CakePHP there are several examples to work with “mocked” objects, especially when emulating some core things, like Session, Email, etc.
This is already nicely covered in the manual.

So what about some real world scenario?

I’d say the first thing that happens very often in the real world, is that you have to retrofit your unit tests to “prove” that an already existing production code is working.
In other words, you don’t write the tests up-front, but you need to write tests for an existing production/application code.

Let’s look at an example… we can imagine a random model with a method like this one:

public function checkIt($id = null) {
        $provider = $this->getProvider($id);

        if ($provider) {
            $info = $provider->getInfo();

            if ($info == '200') {
                return 'got 200';
            } else {
                return 'did not get 200';
            }
        }

        return false;
    }

If we are about to write a unit test for this method, then the first thing we have to figure out is… just what is $provider.. ?
Looking at the code it seems to drive all the little logic in this method.

Once we have a provider, we’d call its method getInfo() to see what sort of response we get back.

This seems simple enough, but what if our $povider needs a bunch of keys and access to a certain IP-restricted area and even to getInfo(), it costs $15 per query? :(

We can’t test or utilize the real $provider Class/Object… so this is a good time to just “mock” one up.

If we look at the code that needs to be tested, we don’t really care how $provider works internally. (Yes, it should be tested also.. but this test has nothing to do with internal workings of the $provider).
Anyway, we do care that if we call the getInfo() method it should return ‘200’. This is the important logic of our application, which we are trying to test.

Sounds good, so if we can fake a return value of ‘200’, we can presume that the provider is doing what it is supposed to do and our unit test for the checkIt() method is done.

Let’s take a look at our unit test:

public function testCheckIt() {
  $provider = $this->getMock('OurProvider', array('getInfo'));
  $provider->expects($this->any())
            ->method('getInfo')
            ->will($this->returnValue('200'));
  $result = $this->Account->checkIt(1);

  $this->assertEquals('got 200', $result);
}

We just mocked up a simple $provider object. We gave it one method: getInfo().
$provider->expects($this->any())->method(‘getInfo’), basically tells our test that we are going to run this method. (If we look back to the model code: $info = $provider->getInfo(), then yeah this is exactly what is supposed to happen).

Of course we also expect a value of ‘got 200′ to be returned by our method.

In our mocked object, we ensure this by doing ->will($this->returnValue(‘200′));. As far as we are concerned as long as ‘200’ is returned and the if ($info == ‘200’) of our model’s method is evaluated correctly, then we are happy.

We expect the return ‘got 200′; to fire correctly in our unit test $this->assertEquals(‘got 200′, $result);. If we are going to run this test right now, it would unfortunately (fortunately) fail because the $provider object used by the model’s code is relying on the actual one, which probably has no proper way of running the getInfo() method.

So far, so good… but there is one obvious problem with our code and the test. The current model code uses the $provider that’s part of some model logic… and not at all the little object that we had mocked earlier.

Now then, we fall back on the technique, which I think is called Dependency Injection. This is exactly where some people have a problem with a tight coupling between unit tests and business logic. Albeit, harmless, there is still a few “extra” lines of code that we have to add to our model to properly test it.
Basically we have to “inject” our mocked $provider object.

public function checkIt($id = null, $provider = null) {
        if (is_null($provider)) {
            $provider = $this->getProvider($id);
        }
        if ($provider) {
            $info = $provider->getInfo();

            if ($info == '200') {
                return 'got 200';
            } else {
                return 'did not get 200';
            }
        }

        return false;
    }

Now we can pass the $provider object to the model’s checkIt() method…
Yeah, we’ll modify our test and make sure that we pass our mocked object to properly test the code:

public function testCheckIt() {
    $provider = $this->getMock('OurProvider', array('getInfo'));
    $provider->expects($this->any())
         ->method('getInfo')
         ->will($this->returnValue('200'));
    $result = $this->Account->checkIt(1, $provider);

    $this->assertEquals('got 200', $result);
  }

The test should be passing now.

I personally feel that the use of objects in such manners actually helps to properly test, but more importantly refactor the code. If we see that a Dependency Injection is required, for example, we have a few things to consider.

  1. Do we have good tests for our $provider
  2. Can we avoid Dependency Injection and possibly refactor the model code?

Improved cache and sessions storage with CakePHP and Redis

CakePHP 2.3 + Redis 2.6.7

For the vast majority of applications out there the issue of cache and session store doesn’t really come into play until the application is large enough to where it needs to be scaled to multiple web servers.
At that point filesystem (default cache and session store) becomes inadequate for a couple of reasons:

  1. You need a central location for sessions because maintaining user sessions on individual servers will be extremely painful. You’d have to persist the user to the same web node.
  2. If you clear/invalidate the cache, it has to be done on all servers consistently or your application will be in a very unpredictable state.
  3. Writing to filesystem is relatively slow.

Often a simple enough solution is to use the database to keep track of sessions across all web servers. This option is nicely integrated into cake, and doesn’t require much in terms of configuration (other than changing the default value from “php” to “database” in core.php).

Today we’ll look at setting up Redis to be used as a cache and sessions storage. Although we are not going leverage some of the advanced features, the benefits of having a very fast, memory-based store with out-of-the-box persistence are great enough for any application, whether you run off of one server or twenty five.

Now that we know “why”… let’s see “how”…

First, we are going to install Redis server. For those on Ubuntu the installation process is honestly a little too easy:

apt-get install redis-server

Done.

Working with default config settings is recommended for a basic installation. (Cake will actually expect certain Redis defaults).

Now that we have our server installed and running, we need to tie PHP and Redis by using any of the available libraries.
The one I like is:
https://github.com/nicolasff/phpredis

The installation is rather simple also.
Clone the repository and run the three commands provided at the link above.

Finally, let’s configure our application to use Redis. All we have to do is modify core.php

First, change the value of $engine to Redis

$engine = 'Redis';

Next, let’s create a separate cache config for our sessions.

Cache::config('session', array(
    'engine' => $engine,
    'prefix' => $prefix . 'cake_session_',
    'duration' => $duration
));

Now we’ll tell cake to use the cache store for sessions as well. The “handler” key ties everything together.
(Ability to use cache store for sessions is an awesome addition to CakePHP and just as well you could use APC, memcached, or some other solution).

Configure::write('Session', array(
        'defaults' => 'cache',
        'timeout' => 100,
        'start' => true,
        'checkAgent' => false,
        'handler' => array(
            'config' => 'session'
        )
    ));

With all of this in place, we can now test our application to make sure everything is working as expected:

  1. CakePHP core Redis Engine tests should be passing
  2. No cache or sessions files are written to local file system
  3. We can run redis-cli and then “KEYS cake*” to see all cake-related keys in the data store. There should be cake_session_*, cake_model_*, cake_core_* keys listed. (This presumes you’ve tried to use the app by clicking around and possibly logging-in)

Intercepting “add to cart” action in Magento

(Nuts and bolts of Magento? Maybe…)

Magento EE 1.11

Once in a while we’d like to intercept Magento’s “add to cart” event and do some fun things with that. Maybe apply a discount or add a specialty product, or as I’ll show in this case, block a user from adding a product to the cart.

As I usually like to show specific examples, let’s consider the following problem:
Using Magento’s EE customer segments we’d like to ensure that a person can purchase some “restricted” product only once, while they are in the given segment.
If they try to add this restricted product to the cart again, while being in our previously-designated segment, we’ll block that action and redirect them back to the cart with an error message.

The best way to handle these sorts of tasks is by employing Magento’s Event/Observer pattern.

First, we need to listen to (or properly phrased “Observe”) an “add to cart” event. In Magento it happens to be checkout_cart_product_add_after.
Shouldn’t we listen to the “*_before” event? Well, yeah… but in the often-crazy Magento world the checkout_cart_product_add_after event actually occurs before the product is added to the cart. (My only guess at this bizarre naming convention, is that the above event is dispatched after the Cart Model is called, but before the cart is actually saved.)

So with that in mind, we’ll adjust our module’s config.xml to tell Magento that we are ready to listen to the above event:

<checkout_cart_product_add_after>
  <observers>
    <teknoid_catch_standard_add_to_cart>
      <type>singleton</type>
      <class>Teknoid_Coolmodule_Model_Observer</class>
      <method>catchAddToCart</method>
    </teknoid_catch_standard_add_to_cart>
  </observers>
</checkout_cart_product_add_after>

Next we’ll need to create an Observer with the catchAddToCart method.

Let’s take a look at that:

public function catchAddToCart($observer) {
        //getting product ID from event data
        $productId = $observer->getProduct()->getId();

        //get product's attribute set ID. (Our designated attribute set ID is 10)
        $attSetId = Mage::getModel('catalog/product')->load($productId)->getAttributeSetId();

        //get current customer
      $customer = Mage::getSingleton('customer/session')->getCustomer();

        //get website ID
      $websiteId = Mage::app()->getStore()->getWebsiteId();

        //only have one segment for testing (ID is 1)
      $segmentId = 1;

        //let's get all segments for this customer ID and website ID
        $segments = Mage::getModel('enterprise_customersegment/customer')
                ->getCustomerSegmentIdsForWebsite($customer->getId(), $websiteId);

        /**
         * If customer is in segment of "designated" and is trying to add another
         * "restricted" product (based on attribute set ID) we block "add to cart"
         * and redirect back with an error message.
         */

   if(in_array($segmentId, $segments) && ($attSetId == 10)) {
            //set error message in session
            Mage::getSingleton('core/session')->addError('Sorry, you cannot add this product to cart');

            //get URL model for cart/index
            $url = Mage::getModel('core/url')->getUrl('checkout/cart/index');

            //set redirect
            Mage::app()->getResponse()->setRedirect($url);

            //send redirect
            Mage::app()->getResponse()->sendResponse();

            //block further action
            exit;
        }
    }

I’m going to keep this post purposely light on details, as there are countless tutorials out there explaining the Event/Observer and configuration setup of Magento.
The code is also pretty straight-forward and well commented, so I’ll just summarize the outcome:

  • We’ve created an observer in Teknoid/Coolmodule/Model/Observer.php (This matches the namespace designation in config.xml)
  • We’ve added a method catchAddToCart() as defined in configuration
  • We are loading the product recieved by the observer
  • We are checking if the current customer is in our “designated” segment and our “restricted” product matches a given attribute set.
  • If so, we set the error message to the session and redirect the customer back to cart

That’s it for now. I hope to publish a couple more posts in the near future about Magento’s Event/Observer strategies and some examples of the things we can do using this powerful way to extend Magento’s functionality.

IDE’s are a thing of the past, and ST is here to save the day

I love sublime text.

I’ve been an IDE user for years. NetBeans, Eclipse, Komodo, phpStorm, phpED, Aptana… I’m sure a few others. I’ve given them all a fair amount of trial over many years.
It seems that because of the nature of what IDE is supposed to be, they are inherently subject to be “bloated”. They can be pretty fast, and many of them I really liked (NetBeans has been my favorite for a while). Still, all of these IDE’s pack a lot of features and lose out on the subjective speed and light feel. (I’ll talk about that later).

Most of my day is spent decoupling things, REST frameworks are kicking ass, everyone is trying to be thinner, lighter, faster, otimizied, minified, cached…

Then we get back to an IDE which tries to couple source control, DB, web services, editor (and all the fun and complex features that come along with it), debugger, a built in web server (?)… well that’s a long list already.
Some IDE’s are good at many things, but aren’t excellent at any one. Some are so good, they just cannot be reasonably fast.

So what did I do?

If the main focus of the day is coding, I, definitely, need a great and fast editor. I’ve used vim, textmate, coda, notepad++, aforementioned IDE’s and finally sublime text.

Sublime “feels” fast.

It’s a subjective matter… I cannot tell you how long it takes in milliseconds to complete a certain task, but the fact that I can access any file in the current project almost instantly, by following some easy keyboard shortcuts, makes the whole experience seem seamless, pleasant and fast flowing. Things never “hang”, but instead respond fast and quite naturally. It kind of does what you expect it to do. And does it elegantly.

Code auto-completion.

This is always tricky when dealing with frameworks and complex applications. I was able to achieve what I need with a few plugins, which is a really nice extensions system for ST (sublime text). Cannot say that it’s perfect, but I may find better ways to utilize some plugins and so far the issues are negligible. My requirements were simple however, as long as what auto-complete suggests makes sense and it nicely stays out of the way, I’ll be happy.

UI

I always thought that I am simply not getting something, but random buttons and icons of IDE’s had always bothered me. The infamous search button of Eclipse is just sad. Also, I don’t really understand if some developers really click on a designated button to indent a bunch of text (or perhaps I’m not getting the purpose of the button), but either way… it’s an overkill that I can live without.

UX

That being said, we are focusing on a code/text editor (for the most part). Paragraph and font formatting is rare and should never be in the way, but text and code editing must be the main focus… and that’s where ST shines!
Multi-line editing, speed and focus on basic features like formatting and selection is what makes ST so easy to love. Everything is naturally and easily accessible through some shortcuts. Sure, you can do all the same stuff with the mouse, but if you spend most of the time with your fingers on the keyboard the keyboard shortcuts are so much faster than re-adjusting your brain and limbs for another input device … a “mouse”…

Learning

Yep, you have to learn some shortcuts to be most effective. (Although, that’s true for any editor or IDE). ST makes it almost a natural desire to learn the shortcuts. The interface makes you think “How can I do this faster?”. What’s best, is that usually there is a good and logical answer.
After a week or so of usage you should be flying through multi-line editing, file searching, complex replacements, find and select features, etc., etc.
Then you’ll have loads of fun picking the right set of plugins and features to make your experience perfect just for you.

JSON

Pretty much everything in the editor, including all the plugins, is configured by editing some JSON files. We all like JSON syntax because it makes everything very clear, human-readable and quite simple. Not to mention, the granularity of “default” and overrides of “user” settings is effective and just makes sense. It is an excellent way to extend and customize ST to your needs.

I like to have everything integrated

If this is your stance, then I guess ST is not going to suit your needs. I do not suggest to overload ST with plugins to mimic an IDE. A minimal set of plugins is great to accomplish things and stay behind the scenes. KISS. That’s the whole point of ST.

Do you tab between apps?

I often switch from console to IDE; although in NetBeans it was very easy to use the built-in terminal… yeah… but in Ubuntu I already like “Terminator”, which is the tool I use for my console-based stuff. I know some shortcuts there and I have things setup the way I like. I don’t really need another tool to do this job. To me, that’s where an IDE loses its value a little.
As far as DB management, I can kind of say the same thing. I’ve yet to find an IDE which also has an excellent DB management features…
To be fair I can’t really say that your typical MySQL clients are all that great (and no, using command line is only good for very quick tasks). Well, even then, what happens once you start using MongoDB, for example?

What about Windows?

I really like that ST looks, feels and works the same on all platforms. IDE’s are generally more popular on Windows (and while Visual Studio is actually nice), most Windows IDE’s are just a way to tie things into an uncomfortable OS and add some elements dictated by the vendor. Customization is minimal and while the IDE has certain power you lose your freedom of choice of the tools. Again if integrated-everything is what you need (for example in .NET development), then clearly we have a distinction in the type of environments typically required for open-source and PHP/JS/Ruby/Python development with emphasis on git, console-based tools (capistrano, sniffers, crons, bake, rake, fake, etc.).
Windows generally lacks a pretty terminal, good console-based tools, things were (and still are) pretty cludgy when it comes to system paths and all. Permissions and package management is not even considered when developing. That’s where the IDE’s picked up the slack in Windows (Similarly to bundled packages like wamp, xampp and friends. I think for Windows they are perfect).
Yet even on Windows even the best plugins for popular IDE’s aren’t quite as nice as TortoiseSVN.

So IDE’s are pointless?

On nix* based systems and Mac’s we have awesome tools that already do the required job. They are typically well designed and well suited to do their main task. Considering that over the years most of them have matured and the developers were using them in conjunction with IDE’s quite often anyways. IDE’s, just like any bulky systems are becoming a thing of the past. Scaling vertically by bunching everything into one basket is not the modern approach. More baskets — more stuff.

Decoupling

By decoupling your editor, console app, browser, version control, DB client you are likely to improve personal performance, because you will be using specific tools that can help you accomplish specific tasks. Find a good workflow and patterned approach when switching between them as needed. If you learn a few tricks in ST your coding and editing productivity alone will increase dramatically and to me that’s a reason alone to use it.

To answer the question above the IDE’s are a dying breed because we are all moving away to lighter and faster technologies. Development environment should be no exception.

p.s. Just a quick note that ST uses the concept of projects, which some other text editors simply lack, and makes the use of them in a very simple and effective way… once again.

Million dollar site in CakePHP …

Is that possible? Yes, and it has been done.

But…

— CakePHP is slow. I’ve seen bench-marks on “Hello world” and cake falls behind other frameworks.
— What’s your caching strategy?
— [crickets] …

  1. Cache content
  2. CakePHP comes with a built-in view caching mechanism. Granted there is always a question of real-time data vs. performance, but I’ve yet to come across a project where at least some content could not be cached. Even a five or ten minute cache can be quite helpful if you’ve got millions of hits a day on your site. I’ve described some strategies in a post a while back, so give it a read, if you are after the details of implementation.
    If you need something faster, consider adding a front-end cache solution, like varnish or reverse-proxy nginx. (If you have a choice consider replacing apache, which requires a lot of tweaking with nginx as your web server, which is quite fast out of the box).

  3. Cache DB queries
  4. Optimizing queries is important. You should always use containable and limit the fields being returned by each query. This optimization will only take you so far, however… and I would argue that before you go head-first into nitty-gritty details of each query, you should invest into setting up memcached (nicely supported by cake) to alleviate some load on your DB.

  5. Index your DB fields
  6. Do you have some JOIN’s? Do you perform some searches on certain fields? (i.e. “username”). Then you’d better remember to index any fields in your DB that are used in the JOIN or being searched on. There are a few ways to properly use indexes, but one thing is for sure — without them, your DB and your app performance is going to suffer greatly.

  7. De-normalize your DB structure, better yet use an appropriate DB system
  8. Speaking of JOIN’s, no matter how you slice it… they are costly for the performance. Sometimes it helps to de-normalize your DB structure in order to avoid such expensive operations. An example would be a users table and user_profiles table. I love to keep minimal information in the users table, but if I find that I keep JOIN’ing user_profiles to get additional information, it is probably time to consider de-normalizing data and move whatever piece of info I need into the users table to avoid an extra JOIN. (Counter cache is another prime example of this).
    That being said, if you find yourself de-normalizing your DB quite heavily, perhaps it is time to consider an alternative to RDBMS. Of course, I am going to suggest MongoDB. Where applicable, it is quite alright to use a mixture of DB systems… always use the right tool for the right job (simple, but powerful statement).

  9. Create read/write DB replicas
  10. Do evaluate your read and write queries. Do you have a ton of admin features, which require complex find() operations? Offload them to a read replica, so that your users (or front-end) write queries do not get in the way. Increasing performance through replication is a nice trick, but you should be cautious not to offload mission-critical data to a replica, because the data might be a little behind as compared to your master server.

  11. Offload heavy tasks to a background process
  12. I do hear this pretty often… “I need to export data to an Excel, and it is taking forever”. This is a perfect example of a job that can be offloaded to a background operation. There is simply no need for a user to sit in-front of the screen and wait for X minutes for the Excel file to build and download. When a user requests an Excel report, add this task to your job queue manger and notify the user when the job is complete. (Have you looked at gearman?).
    — Well, I need to have this real-time.
    — Sorry, but this is not going to happen. Having a user sit and wait is already not “real-time”, IMO. Not to mention the unnecessary stress this kind of operation will put on your DB. (This kind of requirement is a perfect time to consider replication as well).

  13. Use AJAX, when needed
  14. The whole point of AJAX was to minimize the number of heavy requests to the server. Imagine you have an e-commerce site, where you have a page of most popular t-shirts (purchased by the users in the last week) cached… for a week. Makes sense, you only rebuild this list once a week, based on the updated data in the DB. For a week, this popular page on your site is served statically, just like good ol’ HTML.
    But there is a gotcha… you have shopping cart info also as part of this page. Well, AJAX to the rescue. While the rest of the page is cached, one little div, which has the shopping cart summary is easily updated by AJAX.
    Granted this is a very simple example, but think about how well this applies to other situations where you need to mix and match static and dynamic content.

A few other things worth looking into: APC, hiphop-php, Yahoo! performance rules.

Overall, I suggest not to get too hung up on trying to squeeze milliseconds out of your PHP code, unless you’ve exhausted all other resources. Spend your time on proper architecture and caching strategies. Avoid premature optimization and do not trust “hello world” benchmarks when measuring something so complex as an entire framework.
Oh, and before you invest hours into setting up various caching and optimization tools, do not forget to run a load and stress tests to help you identify early bottle-necks upfront.

Would love to hear your experience with optimization, caching and improving performance.

CakePHP + MongoDB, Next Steps

CakePHP 2.2 / MongoDB 2.0.4

Let’s continue building on top of what we’ve done previously.

I’d say that the main consideration when it comes to building out your MongoDB is the choice between embedding everything into a single collection vs linking (or manual referencing) to another collection.

What does it mean?

If we take our blog example, we saw how easy it was to build a tiny app, which allows to add posts. No need to create any tables (collections) or define any sort of schema. Just get the data and insert.
Now we are ready to move forward and our application would probably require an ability to add comments. Because MongoDB is schema-less, first thought would be to store the Post and Comments into a single document. After all, this is the beauty of NoSQL, we don’t need any JOIN’s, we don’t have to worry about belongsTo or hasMany. Again, a simple head-on approach would be: get the data and insert it into the collection of documents (i.e. Posts and Comments are stored together, aka “embedding”).

This is all fine and dandy, and will likely work quite well for our simple example.
However, this post would be useless if we didn’t consider some other issues. For instance, it would become a bit more difficult (actually it can become very difficult in a more complex application) to single out a specific comment. Perhaps you need to edit one comment out of a hundred or so. Having Comments embedded with the Posts, will require a bit of hacking around. (I strongly urge you to read this wonderful post, which details some these considerations: http://seanhess.github.com/2012/02/01/mongodb_relational.html).

So, for the sake of the example (at least), we’ll separate our collections into something more RDBMS-like and create Post and Comment separately (aka “linking” or “manual referencing”).
As you probably suspect we’ll use the same approach as we would in a traditional SQL DB, and create a reference field in the Comments collection… post_id anyone?

Side note: the referencing that we are doing here is called “manual referencing” in MongoDB terms. There is also an option to use DBRefs, which a slightly more advanced approach. In most cases, and such as the one we have here, “manual referencing” is the recommended way to structure data. Also, at the time of writing PHP’s MongoDB driver does not yet support DBRefs, it is however in the works.

With all this mind let’s take a look at our CakePHP view, app/View/Posts/view.ctp:

<?php if (!empty($post)) : ?>
  <h3>
    <?php echo h($post['Post']['title']); ?>
  </h3>
  <p>
    <?php echo h($post['Post']['body']); ?>
  </p>

  <?php if (!empty($comments)) : ?>
    <?php foreach($comments as $comment) : ?>
      <p>
          <?php echo h($comment['Comment']['user']); ?>
      </p>
      <p>
          <?php echo h($comment['Comment']['body']); ?>
      </p>
    <?php endforeach; ?>
  <?php endif; ?>

  <?php
    echo $this->Form->create(array(
      'url' => array(
        'controller' => 'comments',
        'action' => 'add'
      )
    ));
    echo $this->Form->inputs(array(
        'legend' => 'Add some comments:',
        'Comment.user',
        'Comment.body'
        )
    );
    echo $this->Form->input('Comment.post_id', array(
        'type' => 'hidden',
        'value' => $post['Post']['_id']
    ));
    echo $this->Form->end('Add comment');
  ?>
<?php endif; ?>

Nothing different here from our typical CakePHP view. The page will display a post, some comments (if available) and a form to add more comments.
As evident from the form, we should create a Comments Controller with an add() action. To allow the saving of the comments.

Let’s do so right now:

<?php
class CommentsController extends AppController {
       
public function add() {
        if ($this->request->is('post')) {
            if ($this->Comment->save($this->request->data)) {
                $this->Session->setFlash('Your comment has been saved.');
                return $this->redirect(array(
                    'controller' => 'posts',
                    'action' => 'view',
                    $this->request->data['Comment']['post_id']
                ));
            } else {
                $this->Session->setFlash('Unable to add your post.');
            }
        }
    }
}

This is all that we would need in order to start saving Comments and their link (via post_id) to the actual Post. Again, since we are dealing with a schema-less DB there is no need to worry about creating anything on the DB layer. Our Comment model is also not required, because we don’t have any validation logic or custom methods, so an App Model instance, which is created by cake for us is sufficient for this basic operation.

Now, that the comments are saved we should also display them on the “view post” page. If you take a look at the view above, you’ll see that we’ve already setup the logic to display the comments and we’d expect a $comments data array to do so.

Here’s our view() method in the relevant Posts Controller:

public function view($id = null) {
      $this->Post->id = $id;
      $this->set('post', $this->Post->read());
      $this->set('comments', ClassRegistry::init('Comment')->find('all', array(
          'conditions' => array(
            'post_id' => $id
          )
      )));
    }

Whoa… what happened to find(‘all’)? Why did I just issue two separate queries, and had to instantiate a Comment model like that?
Before you despair, let’s summarize a few things:

  1. We did not setup any models, so our relationship between Post and Comment is unknown.
  2. Currently the MongoDB driver does not support relationships as you would expect from a typical RDBMS. This is not to discourage you form the beauty of using an ORM. On the contrary, MongoDB is not meant to have deep-linking and complex relationships between collections. Remember, MongoDB is all about de-normalization. (That being said, there’s no reason why cake can’t support a basic one-level association, the driver is simply not there yet).
  3. Considering the above, this is not to say that we don’t need models at all. I always mention that all business logic should be tucked away in the model as much as possible. So this is not an excuse to make our controllers fat. The example is simple enough, where extra code would be wasteful.

I’m going to wrap things up at this point, as this should already give you some interesting ideas and food for thought. You’ll notice that I’ve embedded the Poster’s name with the Comment. Does that mean that I would take the approach of storing all user data with the Comments? No, not really. Here it is just a display name, which can be derived from the form or a session if we had logged-in users in the system.
Further considerations and examples of this will be a good topic for another day.

CakePHP + MongoDB Introduction

CakePHP 2.2/MongoDB 2.0.4

We all know that CakePHP is awesome, and I’m sure you’ve heard that MongoDB is pretty awesome as well.
So how do we make these awesome technologies play well together?

In an awesomely easy way :)

Let’s get things rolling by setting up MongoDB.
(Instructions for Ubuntu)

sudo pecl install mongo

Add the following line to your php.ini file (on my system in: /etc/php5/apache2/php.ini)

extension=mongo.so

This sets up PHP support for MongoDB.

Let’s install the actual DB, which is easily available through apt-get

sudo apt-get install mongodb mongodb-server

Restart things:

sudo /etc/init.d/apache2 restart

… and double-check that mongo is up and running.

mongo --version

You should see something like: “MongoDB shell version: 2.0.4″

Now, we’ll grab the MongoDB driver for CakePHP, courtesy of Yasushi Ichikawa (ichikaway).

First, cd into your app/Plugin directory.
Then:

sudo git clone git://github.com/ichikaway/cakephp-mongodb.git Mongodb

Once the cloning is complete you should cd into Mongodb directory and:

git checkout cake2.0

Now you have all the components installed and running, and it’s time to try out a simple app.
There is certain beauty in MongoDB, because it doesn’t require you to have any schema. Just write some code, and things will happen magically.

Indeed we’ll just edit our app/Config/database.php

public $default = array(
    'datasource' => 'Mongodb.MongodbSource',
    'database' => 'blog',
    'host' => 'localhost',
    'port' => 27017,
  );

Add to app/Config/bootstrap.php

CakePlugin::load('Mongodb');

Create a model:

<?php
class Post extends AppModel {
 
  public $validate = array(
    'title' => array(
        'rule' => 'notEmpty'
    ),
    'body' => array(
        'rule' => 'notEmpty'
    )
  ); 
}

Controller:

<?php
class PostsController extends AppController {
   
    public function index() {
        $this->set('posts', $this->Post->find('all'));
    }
     
    public function add() {
        if ($this->request->is('post')) {
            if ($this->Post->save($this->request->data)) {
                $this->Session->setFlash('Your post has been saved.');
                return $this->redirect(array('action' => 'index'));
            } else {
                $this->Session->setFlash('Unable to add your post.');
            }
        }
    }
}

And a couple of views…

app/View/index.ctp

<?php
  foreach ($posts as $post) {
    debug($post);
  }

app/View/add.ctp

<?php

echo $this->Form->create();
echo $this->Form->inputs(array(
    'title', 'body'
));
echo $this->Form->end('Add');

Congratulations! Your first CakePHP/MongoDB app is now ready to rock.

You can go ahead and try adding some posts, and see them appear (albeit only as debug output) on the index action.