May 072012
 

Disclaimer: This isn’t going to be about php, so if that doesn’t tickly your fancy feel free to skip this entry. It’s also a bit of a rant.

Every now and then you figure out you want to do some cool css trick or update some css or markup that’s been lagging behind. Last friday I was in total weekend mode so I figured I’d update some of the old markup and css in the project I’m currently working on. The header area in that project was one of the first pieces of html and css created when we started developing it, and it has been left largely untouched since those early days in 2009. It was in desperate need of some tender CSS love.

The first thing I wanted to fix was the header background. It’s got a gradient background image, with a logo that has the gradient as background color. Depending on which browser you’re using, the background image and the logo is always one or two pixels off, because that’s just the way the markup is. We still have to support relatively old and obscure browsers like IE7, so keeping backwards compatibility is important to not unnecessarily annoy our customer(s). So, since we’re living in 2012 nowadays it should be as easy as adding a css background gradient with a fallback to the background image:

background: url("/images/navbar_top.png") repeat-x; /* legacy background image */
background: linear-gradient(to bottom, #434A50 0%,#2B3036 80%); /* W3C */

Yeah, not so much. The CSS level 3 specification for background gradients is still not finished and all browsers that has implemented it so far has done so based on early drafts, their own interpretation of the specification as well as their own desires about how the specification should work. According to the MDN page about CSS linear gradients, achieving this effect across all browsers is not at all trivial.

First of all: the background gradient CSS3 specification is of course not yet final and it is still subject to changes. Also, every browser on the planet has its own way of specifying background gradients, and they’re all non-compatible as well as wildly different. If that wasn’t enough to completely turn you off CSS gradients, bear in mind that each browser also has different browser-/vendor-specific implementations depending on the browser version, as the recommended implementation has been changed quite a lot since Apple introduced this feature back in 2008. To end it all, the currently final version of the CSS3 recommended standard has switched around one of the keywords – instead of specifying where you want the gradient to start (“top”), you specify the direction (“to bottom”) – making the final recommendation inversely compatible with the currently available browser-specific implementations. So, to implement something as trivial as a background gradient you need to take into consideration several different versions of a multitude of browsers, as well as a totally moving specification. With that in mind, here is the bit of CSS I ended up using for that header background gradient:

background: url("/images/navbar_top.png") repeat-x; /* legacy background image */
background: -moz-linear-gradient(top, #434A50 0%, #2B3036 80%); /* FF3.6+ */
background: -khtml-gradient(linear, left top, left bottom, color-stop(0%,#434A50), color-stop(80%,#2B3036)); /* KHTML */
background: -webkit-gradient(linear, left top, left bottom, color-stop(0%,#434A50), color-stop(80%,#2B3036)); /* Chrome, Safari4+ */
background: -webkit-linear-gradient(top, #434A50 0%,#2B3036 80%); /* Chrome10+, Safari5.1+ */
background: -o-linear-gradient(top, #434A50 0%,#2B3036 80%); /* Opera11.10+ */
background: -ms-linear-gradient(top, #434A50 0%,#2B3036 80%); /* IE10+ */
filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#434A50', endColorstr='#2B3036',GradientType=0 ); /* IE6-9 */
background: linear-gradient(to bottom, #434A50 0%,#2B3036 80%); /* Current W3C recommendation */

I kid you not.
Luckily, with this blob of CSS we’re able to support pretty much all browsers in use today. Yay for us! Only marginally smaller than the byte size of the image we’re currently using!

I’ll follow up with a new post soon, with more juicy CSS fun. Comments are much appreciated!

Feb 282012
 

Timezone handling can sometimes be a bitch. Turns out, using a couple of PHPs classes and functions, it’s quite easy.

Let’s say you have a web application with the following scenario:
Users from around the planet, and your server is located in the UK.

You would initialize your DateTime object like this:

$timezoneUTC = new DateTimeZone('UTC');
$dateTime = new DateTime('2012-02-23 10:22', $timezoneUTC);

echo $dateTime->format('Y-m-d H:i e').'<br/>';

This should output:
2012-02-23 10:22 UTC

For an international user, seeing his or her local time would be nice.
To do this, simply change the timezone of the existing DateTime object, like this:

$timeZoneSweden = new DateTimeZone('Europe/Stockholm');
$dateTime->setTimeZone($timeZoneSweden);
echo $dateTime->format('Y-m-d H:i e').'<br/>';

$timeZoneNY = new DateTimeZone('America/New_York');
$dateTime->setTimeZone($timeZoneNY);
echo $dateTime->format('Y-m-d H:i e').'<br/>';

$timeZoneBKK = new DateTimeZone('Asia/Bangkok');
$dateTime->setTimeZone($timeZoneBKK);
echo $dateTime->format('Y-m-d H:i e').'<br/>';

This should output:
2012-02-23 11:22 Europe/Stockholm
2012-02-23 05:22 America/New_York
2012-02-23 17:22 Asia/Bangkok

The final code should look something like this:

$timezoneUTC = new DateTimeZone('UTC');
$timeZoneSweden = new DateTimeZone('Europe/Stockholm');
$dateTime = new DateTime('2012-02-23 10:22', $timezoneUTC);
$dateTime->setTimeZone($timeZoneSweden);

This will leave a DateTime object set with the users timezone.

And that’s it.

Oct 202011
 

With the recent massive flood of frameworks, libraries and toolkits on the market these days it is easy to forget that underneath it all is the good old, plain and simple, PHP with all its kinks, quirks, and huuge set of builtin functionality.

PHP has vast amount of extensions which solve all sort of problems. And if PHP doesn’t have it built-in, we have an impressive amount of additional extensions both on pecl and now recently more and more on github.
There is a high chance that someone else has been in your shoes already and solved the problem, so it is worth looking around over the horizon and see if the problem has been solved already.

For some reason the current practice seems to be the “RoR” idiocy where “RoR developers” barely even know that there is this Ruby some miles down the stack. PHP has hit this “stepping stone” already with WordPress, Drupal and even Symfony and that is a weird and scary thought. Remembering “where you came from” is an important fact to remember, even for those who specialize in specific products. Looking at how other projects work, comparing notes, work ethics, features and functionality is also very important. Getting different perspective and knowledge is how we can improve our solutions and work more efficiently. If your specific product doesn’t have native support for something, why not look at a different framework/library/cms/toolkit/.. even PHP extensions?

As June mentioned earlier, going ‘back home’ and checkout the PHP manual pages is generally a good idea. Things change, manual pages are updated, improved, added, and you have different perspective, other problems to solve and so on. Even though you believe you know all the basics, you still need to practice them, and that includes browsing the manual from time to time, again and again – no matter which project it is.

So what is the best way to stay in touch? Kept up2date with new ways and offerings? New solutions to the same problem? Get involved!

By far the best way is to get involved with the project you are using. Even just silently idling on the mailinglists and read the subjects. Subscribing to the commit lists is a fantastic way to see precisly what is going on and see which direction the project is taking. Who knows, after a while you may spot something the others didn’t. Get an idea for a killer feature. Shed a light on different perspective the others didn’t think of. After a while hanging on the lists you’ll get a feeling for how the project works, and hopefully start chiming in. Give your 2cents, and who knows – even cook up a patch or two.

 Posted by at 4:37 pm
Oct 072011
 

Introduction

Whilst everyone is buzzing and creating fancy new Symfony2/Doctrine 2 applications, and perhaps even a shift to new frameworks/no frameworks, a great deal of us are still maintaining legacy apps and will be for some time to come. As these apps grow, we occasionally need to look back and scream at our old code and wonder why we didn’t make it more scalable or use neat optimisation tricks back when it was first conceived. The fact is, many of these “tricks” are not necessary at the time for a virgin app, and we need to develop code that is relevant to the task at hand.

That said, being aware of some of the case studies I will present to you now may help you to optimise old code, but may also allow you to think twice when you are working with new code – as the things I will describe do not take too much time to implement first time round.

Case study: Working with large resultsets and array_merge

In my sample application, the database contained many different types of organisation stored in different tables, and due to the structure of the data and the criteria for each organisation, there was no easy way to retrieve several organisations at once using pure SQL and joins (well, no convenient way). The solution is relatively simple, one query per organisation then combine the results, in this case as we go along:

$membershipClasses = array("Entity1", "Entity2", "Entity3", "Entity4", "Entity5");
    $results = array();
    foreach ($membershipClasses as $joinedTable)
    {
      $query = $this->createQuery()->from("TablePrefix".$joinedTable." t");
      // Lots of differing criteria based on which class we were dealing with
      ...

      $results = array_merge($query->execute(array(), $hydration), $results);
    }

This logic was used to generate a report with approximately 50,000 rows, used 1.2GB of internal memory and took around 20 minutes to complete – often failing or bringing the system to a halt.

One optimisation used in this case (many more are surely possible but lets focus on one) involves the last line – using PHP’s array_merge function. Simply switching this out to use a simple array traversal brought the execution time down to under 2 minutes, however there was no change in memory usage.


$theseResults = $query->execute(array(), $hydration);
foreach ($theseResults as $aResult)
{
  $results[] = $aResult;
}

Why is this so much faster? And why can’t we use += instead?

 

The “problem” with array_merge, is that even though it discards numeric keys, it still has to *check* them all first, which is a lot of overhead on 50,000 rows. We can’t use += in this case either, because that would respect the numeric keys which would start from 0 each time, and therefore the results would not “stack” as intended, unless we tell Doctrine to index the results with some unique key.

Why it takes *so* much longer is a bit of a mystery to me. I’ve looked at the c code behind it (ext/standard/array.c) and it’s, well quite frankly voodoo.

Case study: Limiting results

Something that was “fixed” (read: removed) in Doctrine 2 was the ability to limit results returned from queries, at least the “magic” part of it. The problem with limiting hydrated results is that you need to know exactly how many rows each sub-tree will contain before you can limit the query as a whole. Imagine a person with many addresses, and you want an array limited to 5 people – you can’t list add “LIMIT 5″ to the end of the query, because when this is hydrated you will most likely end up with one or 2 people, the second of whom may not have all their addresses, because you’ve told your database manager to return 5 *rows*, it has no idea how these rows relate to your model.

In Doctrine 1, adding a limit clause would cause all sorts of magic to happen, the resulting query ending up as a collection of complex subqueries each with their own limit clause, the more joins you introduced, and the more levels of join, the more complicated it becomes. First level joining is not so bad, but joining several levels deep soon starts to get heavy, and your execution speed will suffer for it. Couple this with a large resultset and you will see the smoke drifting from the server room in no time.

So, what’s the solution? Well, this one is not so clear cut – you have to experiment. Sometimes, doing it the “magic” way will work just fine, and is totally acceptable, but when the query becomes “heavy” you have a few options:

Work out your subset first

This is the default Doctrine 2 approach, it’s quite simple – you use one query to get all the ids of the subset of top level elements, then pass these IDs to another query in a “WHERE IN” clause. When you look at the resulting SQL, it can be quite scary, especially if you are dealing with many results – you can easily be saying “WHERE IN (1,2,3,4…..9999999999)”. Most dbms will handle this surprisingly well, as long as you are using the primary keys, so experiment with it and it might be the way to go.


if ($limit)
{
  $subQuery = $this->createQuery("foo")
                   ->select("foo.id")
                   -> //
                   ->limit(50);
  $ids      = $subQuery->execute(array(), DOCTRINE_CORE::HYDRATE_SINGLE_SCALAR);
  $query->andWhereIn("cf.id", $ids);
}

Note the use of “single scalar hydration” – we have no need for more than this (to be really picky could go straight to PDO here but for consistency this is ok). In this case, the single scalar hydration will give us a resulting array of raw IDs, which is exactly what we need to pass to the main query.

Make your own subquery

A variation on the above is to embed the “ID sucking” query into your main query, some database engines may prefer this style, but I have not noticed any significant performance gain or loss doing this so prefer the above option as it’s cleaner in the PHP code.


$query->whereIn('SELECT id FROM my_other_table WHERE blah LIMIT 50');

In a perfect world, this is the kind of thing Doctrine could have done instead of the subquery magic, but there are so many factors to consider here that I can understand why they did not go down that route.

Stick with the magic, but simplify

It is also possible to continue using the magic, but greatly improve performance by simplifying. Remove all those second+ level joins and use separate queries to get hold of the data you need. How often have you added a chain of joins just to get one snippet of data from the last node? Scrap all the joins and make a new query later where you can pass in all the IDs from your main query and get the data you need. Now you don’t need a limit clause, because you are asking for exactly the data you need in the first place.

Start with the fastest hydration mode and work upwards

This one I can’t stress enough – an ORM such as Doctrine is there to give you the tools you need when you need them, but it is often the case that applications are built with all the overhead and magic just because it can be, or because it’s the default behaviour. Stop. Look at what data you *actually* need, and especially ask yourself if you need objects in your results. Perhaps you are retrieving all your users and listing them with their profile data, but you are hydrating them as objects because you need some custom functions you’ve written, classic examples are getAge() or getFullName() which are derived from other fields. If this is the only reason you are object hydrating, consider something like this:


class User
{
  function getAge()
  {
    return self::getAgeFromDOB($this["dob"]);
  }

  function getFullName()
  {
    return self::getFullNameFromUserArray($this);
  }

  public static function getAgeFromDOB($dob)
  {
    $dob = strtotime($dob);

    $year_diff  = date("Y") - date("Y", $dob);
    $month_diff = date("m") - date("m", $dob);
    $day_diff   = date("d") - date("d", $dob);

    if ($month_diff &lt; 0 || ($month_diff == 0 &amp;&amp; $day_diff &lt; 0))
    {
      $year_diff--;
    }
    return $year_diff;
  }

  public static function getFullNameFromUserArray($user)
  {
    return $user[&quot;first_name&quot;] . &quot; &quot; . $user[&quot;first_name&quot;];
  }
}

In this example, we’ve kept the sideways compatibility of the getFullName function with array/object hydration, so it will work whether an object is passed or an array (sanity checking needed of course). This method could be expanded to support more “raw” hydration methods also. Now in our templates, we can just use:

<li><?php echo $user["username"]; ?></li>
<li><?php echo $user["Addresses"][0]["city"]; ?></li>
<li><?php echo User::getFullNameFromUserArray($user); ?></li>
<li><?php echo User::getAgeFromDOB($user["dob"]); ?></li>
<li>...</li>

This can massively speed up your application if you have long lists or are generally dealing with lots of data, especially data than spans several levels.

Lets also take a second to consider the super fast hydration methods, if we are really struggling for resources (perhaps working with large downloadable reports) and we can live with slightly less friendly arrays of data, scalar hydration can save the day. Shifting to scalar hydration means we lose the ability to later instantly switch to object hydration due to the alternate syntax and lack of nesting, but if we are considering scalar hydration in the first place we are probably in a situation where object hydration will never be practical.

<li><?php echo $results["u_username"]; ?></li>
<li><?php echo $results["a_city"]; ?></li>
<li><?php echo User::getFullNameFromUserArray(array("first_name" => $results["u_first_name"], "last_name" => $["u_last_name"])); ?></li>
<li><?php echo User::getAgeFromDOB($results["u_dob"]); ?></li>
<li>...</li>

I could go on and on but I hope these points have given you some food for thought!

Apr 212009
 

In the last few days (evenings) I have been revisiting several games that I found amusing when I was younger. Some of these games I spent endless hours playing – both single- and multiplayer. I also convinced a friend of mine to try playing some of these games with me – apparantly he also enjoyed playing some of them in the past.

As it turns out – these games aren’t as good as they once were. Or – our perceptions, expectations and gaming habits have changed. We want more. The graphical standard – which were breathtaking at the time – now makes the game look old, outdated, boring and uninteresting. The intense gameplay we experienced back then has now turned into boring tasks, repetitive assignments and uninvolving gameplay. The AI is still stunning – although now it is not because it is so good, rather that it is mindnumbingly bad.

Truth be told – I didn’t have as much fun revisiting some of these games as I thought I would.

What on earth does that have to do with PHP? Well, by coincidence, this weekend I also found a backup CD I had lying around – and it had some of my older PHP projects on it. I think you know where I’m going with this now ;)

Yes, you are right – I was amazed by how badly coded these projects were. They are riddled with amateurish mistakes, shortcuts, half-assed solutions and just in general messy code. Functionality that I still remember spending endless hours implementing turns out to be nothing more than semi-intelligent calculations written in poorly designed PHP code.

Notices are being thrown all over the place, short open tags everywhere (and completely at random), $_GLOBALS variables used inconsistently, naming confusion, duplicate functionality in different functions, no classes (well, actually there was one, with one function in it but it wasn’t even used), and no concept of how an application is structured.

Suffice to say I wasn’t as impressed looking at it now, as I was when I was writing it. However – this is a good thing! Spotting obvious mistakes in old code shows that you’ve learnt. I have learnt a tremendous amount of new stuff over the past two, three years, it’s hard to even think about it all!

So – if for nothing else than to give you a good laugh – dig up some old code and have a look at it. Try to remember the good times you had writing it – the challenges you faced and how good you felt when you finally found the ultimate solution. Which turns out was probably not so much the universal ultimate solution, but rather “the best you could come up with, at the time”.