Do Cool Things That Matter

cool_things

I have this magnet on my desk that I think really speaks well to what I try to do on a daily basis at CURE.  As a technologist, I have a strong conviction that awesome tech can be used not just to build billion dollar businesses, but also to help out those in need.  At CURE this means building apps that play a part in healing kids that would otherwise remain disabled, or in some cases, eventually die from their condition.

One of those apps is our offline first medical records solution that plays an important role in our Hydrocephalus program.  Using this app as a launchpad, we aim to do even cooler things that matter.  In particular, I am a part of the hospitalrun project, which aims to build open source, modern software for charitable hospitals in the developing world.  My hope with this project is to leverage the latest and greatest web technologies to build software that helps many organizations throughout the world do things that matter.

Putting cure.org on a diet

As the Director of Technology Development at CURE International, one of my primary responsibilities is the care and feeding of CURE’s website, http://cure.org.

cure.org started out as a WordPress site in 2009 and frankly speaking, it has put on some weight since then.  Our homepage currently loads over 5mb in resources.  Thanks to profiling tools such as New Relic, I can tell you that our average browser page load time is around 8 seconds.  In Internet time, that is an eternity!  Profiling on mobile devices reveals even slower times.

To date, we have addressed performance of cure.org by doing the following:

  1. Combine resources into 1 file to limit the number of requests.  For example, all of our js gets compiled into a minimized js file and all of our css gets compilied into a minimized css file.
  2. Implement a WordPress caching plugin.
  3. Move images to a CDN.
  4. Front the site with cloudflare.

Unfortunately, these methods have not been enough and I believe we need a more system wide overhaul to cure.org to address performance.  I am hoping to document that overhaul through this blog.  At a high level, here is the plan to address performance:

  1. Evaluate all plugins.  We currently have 25 active plugins including some internally curated plugins.  Some initial profiling has revealed that we have several plugins that need to be replaced, removed or improved for performance sake
  2. Optimize home page/other key pages for optimal rendering.  As I mentioned above our home page currently weighs in at 5mb of resources.  The content on that (and other pages) needs to be evaluated to see what we can do to slim it down.  Also, a progressive rendering of these pages would at the very least improve perceived performance.
  3. Implement a theme with a responsive design.  One of the key objectives of this performance overhaul is to address the poor experience our mobile users have with cure.org.  Implementing a responsive design will allow us to address the mobile experience as well as improving performance.  Recently Luke Wroblewski wrote about the Impact of Responsive Designs and we hope that implementing a responsive design will impact CURE in a similar way.

WordCamp Philly Slides

I recently built an offline web app which uses HTML5 features such as Application Cache, IndexedDB, and the FileSystem API.  This web app is a Medical Records System that uses WordPress as a backend.  I recently gave a talk at WordCamp Philly about this project and my slides from the talk, “Surviving the Offline Apocalypse” are now available here: http://resplendentdev.com/wcphilly

 

Lesson learned: hierarchical custom post types don’t scale well

The other day I ran into an issue where an in-house developed WordPress plugin with custom post types was causing an out of memory error when attempting to view the custom posts in the admin dashboard.

It turns out that the issue is caused by the fact that listing hierarchical post types in the admin dashboard does the following (taken from Nacin’s comment here http://core.trac.wordpress.org/ticket/15459):

  • Query all posts
  • Group them based on ID and post_parent
  • Begin rendering posts until we’ve reached the total to display per page

That first bit of query all posts just so happens to load all of the posts into memory.  If your custom post type doesn’t have a ton of posts, this isn’t a problem, but in our case we had 40k custom posts that were causing an out of memory error (eg Allowed memory size of 268435456 bytes exhausted).  The 256MB limit is set by default in WordPress via the WP_MAX_MEMORY_LIMIT constant.

As I examined the plugin in question, I realized that it had no need to be registering its custom post type as hierarchical, so in my case it was an easy fix.  However, if you have custom post types that are hierarchical for good reason, note that you may run into this issue.  There is an enhancement ticket for this issue here:

http://core.trac.wordpress.org/ticket/15459

WordPress uploads and load balancing

I am currently working on a project building out a load balanced WordPress install with multiple web servers and I came across an issue that I think I have found an innovative way to address.  Recently at WordCamp San Francisco, Mark Jaquith in his session Scaling, Servers, and Deploys — Oh My! stated that his preferred way to handle uploads in a load balanced WordPress environment is to use a single server for WordPress admin and media uploads and then use some type of task to push the media out to the other servers.  I think this is great for most load balanced WordPress environments since the load from admins is minimal compared to end users.

However, what happens when you give your end users the capability to upload media?  You certainly don’t want to move your end users to a single server — that defeats the purpose of having a load balancer.

In my case, the project I am working on is being built on RackSpace cloud servers and is using RackSpace cloud files for CDN distribution of static images.  The thought occurred to me that there had to be a way for my cloud servers to somehow share uploaded files.  After googling around I came across CloudFuse.  CloudFuse essentially allows me to mount RackSpace cloud files as a directory on my web servers.  Each web server has a directory that mounts the cloud files and WordPress’ upload path is set to that directory.  Multiple web servers seem to play nicely with uploading files to the directory, though I have not tested it under an type of heavy load.

There is also another benefit to this solution.  Since cloud files include CDN distribution, the uploaded files can then be served using the CDN which circumvents two things:

  1. Having to push the uploaded images to multiple servers and/or a CDN.
  2. The CloudFuse mount only has to be used for write operations.

Admittedly this solution only works if you are using RackSpace, though I believe you could pull off something similar with other CDNs.

My Video from WordCampSF

Recently I had the privilege of speaking at WordCamp San Francisco on “WordPress for the Greater Good”.  It was a shared presentation with Zach Berke of Exygy who spoke on the excellent work his team did for UNICEF in Uganda.

In this presentation I talked about how CURE International (my employer) moved to WordPress to become an award-winning website as well as providing their constituents with the world’s first social sponsorship platform (cure.org/curekids). I also covered the release of Personal Fundraiser, an open source (GPL) WordPress plugin that provides organizations the ability to allow their fans and constituents to create their own custom online fundraisers using Paypal donations and other payment methods.

You can watch the video here:

More details about the presentation are available at the WordCamp San Francisco site.

nginx config for TouchPad WordPress

Here is the nginx config I used for my TouchPad WordPress server:

# Upstream to abstract backend connection(s) for php
upstream php {
        server unix:/tmp/php-cgi.socket;
        server 127.0.0.1:9001;
}

server {
        ## Your website name goes here.
        server_name localhost;
        ## Your only path reference.
        root /usr/local/share/wordpress;
        ## This should be in your http block and if it is, it's not needed here.
        index index.php;

        location = /favicon.ico {
                log_not_found off;
                access_log off;
        }

        location = /robots.txt {
                allow all;
                log_not_found off;
                access_log off;
        }

        location / {
                # This is cool because no php is touched for static content
                try_files $uri $uri/ /index.php;
        }

        location ~ .php$ {
            fastcgi_pass 127.0.0.1:9001;
            fastcgi_index index.php;
            include fastcgi_params;
        }

        location ~* .(js|css|png|jpg|jpeg|gif|ico)$ {
                expires max;
                log_not_found off;
        }
}

HP TouchPad WordPress Server

Thanks to HP’s recent fire sale of their TouchPad tablet, I found myself with a new device to hack.  After digging around webOSroundup, I found that I could install Ubuntu on the TouchPad.  This got me thinking:

I use Ubuntu for WordPress development.  Could I run a WordPress server on a TouchPad?

It turns out that by using nginx, MySQL and php5-fpm the answer is yes:

TouchPad WordPress serverHere is how:

  1. Install Ubuntu on the TouchPad using Preware, Meta-Doctor, and UbuntuChroot.  I followed the directions from the webOSroundup forums.   Once you have Ubuntu up and running do the following:
  2. Install nginx: apt-get install nginx
  3. Before installing mysql, you will need to use dpkg-divert to avoid the error Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused:
    dpkg-divert --local --rename --add /sbin/initctl
    ln -s /bin/true /sbin/initctl

    For more details see here and here.

  4. Install mysql and php-fpm:
    apt-get install mysql-server php5-fpm php5-mysql
  5. Download WordPress latest from wordpress.org and extract locally:
    cd /usr/local/share
    wget http://wordpress.org/latest.tar.gz
    tar -xzf latest.tar.gz
  6. Edit /etc/php5/fpm/pool.d/www.conf and change the listen port to 9001 (9000 is already in use).
  7. Edit /etc/nginx/sites-available/default for use with wordpress. I’ll post my config as a separate post. I used the basic setup detailed here: http://wiki.nginx.org/WordPress, but I did have to tweak it slightly.
  8. (Re)start the necessary services:
    mysql: mysqld_safe &
    php5-fpm: /etc/init.d/php5-fpm restart
    nginx /etc/init.d/nginx restart
  9. Perform the WordPress Famous 5 Minute Install.
  10. Enjoy!