Category Archives: tips

tips

Avoiding the Performance Panic Spiral of Doom

Leave a comment

The following warning applies to anytime you try to fix a misbehaving system without understanding the cause of the problem, but especially relevant when trying to fix performance issues without knowing the cause:

The trouble started when the site started randomly slowing to a crawl at random times. The tech team met to discuss the issue.  Having failed to extract the cause by the act of stuffing enough smart people in a room, the topic shifted to solutions.

“Let’s switch our caching from memcached to redis” I said. The testing went well, and the change was made. The following testing, accompanied by a dose optimistic thinking, let to the conclusion that the issue was improved.
Everyone was happy, until it was discovered that the registration system was broken, because in one specific function, PHP failed to set the redis cache, causing a redirect loop. We fixed the problem, but the performance issue returned.Following this, another dozen configuration and code changes were tried. Since we could not consistently reproduce the performance issue, it was questionable whether any of these changes helped. The only thing which became clear was that our site was becoming increasingly unstable, and we had little experience dealing with all the new components. In desperation, we decided to start over with a new server build.

The testing of the new server went OK, until I decided to throw another new wrench in at the last second – switching from MySQL to AuroraDB. “AuroraDB is 5X faster and 100% wire compatible with MySQL” according to Amazon, but it turns out that the PerconaDB client library on the server was not, the AuroraDB default parameter group is not configured properly for high query rates, and WordPress+mysqli PHP library+AuroraDB don’t play well together.

So now, we had all our existing problems, plus the issue of configuring a new server a new set of management tools, plus the issues of switching to a new database server. Eventually, we solved all the problems we created by either learning to use the new components or reverting to old ones, but we never did figure out the cause of the performance issue, and simply patched it over with more hardware.

What’s the moral of this story?

If a website is suddenly slow, unreliable generally misbehaving for performance-related reasons, 
DO NOT TRY TO MAKE PERFORMANCE IMPROVEMENTS WITHOUT UNDERSTANDING THE ROOT CAUSE
  •  Any performance-related change should be tested to see if it makes things better.  This is impossible without a stable site.
  • Performance-related configuration and code changes should be based on evidence – quantitative proof that the specified change will help.
  • Making changes based on hunches and Internet guides is a potentially endless process as software like MySQL, Varnish, Nginx, etc offer hundreds of parameters with millions of opinions online about what is best.
  • The approach of making optimizations in the dark is a huge time drain when a quick and short-term solution is needed.  You will make many changes with unknown effectiveness, possible falling into the dreaded Performance Panic Spiral of Doom:
    1. Try to fix a problem with guesswork without understanding the cause
    2. Break an unrelated component in the process
    3. Try a more drastic fix to fix both issues
    4. Repeat, until the site is a disaster zone
HOW TO ACTUALLY RESOLVE PERFORMANCE OUTAGES: EVIDENCE-BASED ROOT CAUSE ANALYSIS
  1. Enhance your environmental awareness (by improving your monitoring & diagnostic tools*) until you can visualize, isolate, and identify the problem.
  2. Fix the problem.

* For example, using New Relic, error logs, htop, ntop, xdebug, etc.

tips

Note on iOS (iPhone/iPad) background app energy usage

1 Comment

There is some confusion about the impact of background apps on iOS on battery life. Some prominent sources advise “quitting” all background apps to maximize batter life while others say that background apps are suspended and use no battery.

Actually, both statements are true. The confusing part is that iOS does not visually indicate suspended versus running background apps. Some apps are will obviously use power in the background – GPS navigation, instant messaging, or a music player. In other cases, it is not clear.

While there are many apps in the App Store that claim to help “improve” battery life, Apple does not allow access to any API which can have any impact on battery life. The only way to do so is to jailbreak your iPhone/iPad and install an app like BatteryDetective, which can measure how much power background apps and services actually use.  (It also the only app which can tell you how much charge your iPhone battery can retain.)

In my brief testing, I made a few surprises. When an iPhone is not actively being used (the screen is off), the biggest power drain are the various radios: WiFi, 3G and Bluetooth. So first tip is to turn off any service you don’t need. SBSettings is useful in doing so with one tap:

Here are some sample measurements (not intended to be typical) in milli-amp drain in the background: iPod: 50mA, Gmail App: 185mA, Skype: 265mA. I found that some apps are seeming unnecessary energy-hogs: why does Skype use so much power in idle mode? I also found that the built-in Messages app was using way too much power – which went down after I deleted some of four years of received SMS and MMS messages.

BatteryMeter has a manual mode so you can take samples when toggling background services:

Unfortunately, measuring the power usage of apps and services is not practical for most users. So if you want to maximize battery life, just (1) turn off any function/service you don’t need and (2) remember that when and only when an app is doing anything in the background, it will drain the battery.

tips

Four tips for more effective web design projects

Leave a comment

I want to share some strategies for maximizing the productivity and effectiveness of  web design projects.  I’ve used them when the project has a large scope but minimal budget for a professional design team, such as non-profit organizations.

1: Visualize your users with analytics, personas, and user stories

The most important consideration in web design is to meet the needs of your users. There are two aspects to this:

First, collect statistical data on the demographics and usage patterns of existing users. Use web analytics, (GA), in-page analytics (GA events, crazyegg) to understand exactly how users use the site, what content is popular, and how that relates to revenue goals.  There is no need to rely on hunches and guesses when the relevant information is easily available.

Second: visualize the kind of customers you have and want to have with personas and user stories. Understand their goals and visualize the best way to meet them.

2: Steal ideas from the best

Web design consists mostly of weaving together design patterns from other projects. Coming up with entirely new design paradigms is very rare, difficult and risky. With that in mind, how can we choose the best design patters for a projects? By systematically borrowing from other successful designs.

One of the nice things about web development is that it is much easier to copy successful designs than in other fields. So why not pick the very best websites on the web? There are many websites which have spent millions on market research and high-end design firms. I systematically research successful designs and borrow the best elements from each. Many people make the mistake of only looking at their competition for inspiration.  But looking just like the competition won’t inspire users and limits you to a very narrow sample.

Here are three ways to find sources for inspiration:

  • Look for winners of web design awards
  • Frequent web design blogs & magazines
  • The easiest way: go straight for the websites with the biggest design budgets.

For #3, I sometimes use Wikipedia as a resource to find proxies for design budgets and just go down the list. For example, I use the list of the wealthiest charitable foundations or the largest market caps.  I go straight down the list and note the designs I like.
For example, #5 – the Ford Foundation has a very effective design.  The useful information we can gather goes far beyond visual elements.  By researching their site, press releases and job ads, I was quickly able to identify their technology platform, content management system, the person in charge of their information technology (technical people at non-profits are often willing to share expertise) and some of the contractors they hired to work on it.  The cross-section of people visiting the Ford Foundation and your site is likely to be very small, so you can borrow more liberally than you could from a competitor.

3: Develop a design language

A “design language” is a set of visual concepts which we can use as building blocks for a web design. A design language helps us share ideas visually and then create consistent look and feel for the site, even if multiple people or multiple iterations are involved. A coherent design pattern is very important for a site. I suggest something like a Google Docs document with screenshots of elements from other sites and mockups. Eventually we can weed it down to unique elements representing key building blocks: header, menu, tabular data, form, product list, etc.

A design language is also important for controlling costs, as it is cheaper to pay a designer to create a reusable set of elements than to pay them to create each page from scratch.

4: Aim for the web technologies of the future

The lifetime of the average web design is about three years, though many last much longer. Three years is a long time in web design – paradigms come and go very quickly, as does the need to respond to competitors. That’s why it’s important to find the right balance between targeting emerging technologies while controlling risk. For example, most people don’t have smartphones with full web browsers, but they are likely to in a year or two. It thus makes more sense to invest in full-featured mobile sites and not the last-generation WAP version. At the same time, we don’t know if the Apple, Android, or Windows Phone platform will be most popular in the future, so it’s best to build mobile web versions than native apps in today’s hottest platform.

tips, tutorials

Great image optimization tools for Windows and Mac

1 Comment

Over the last few weeks, I’ve experimented with image optimization tools. Using these tools, I have rapidly eliminated gigabytes of image data from thousands of images without any quality loss. Over time, this should translate to many terabytes of bandwidth savings.

Because these tools can be run in batch mode on thousands of images at a time, they are useful for optimizing large, existing image libraries.   They are lossless and designed for bulk mode, which means you can safely run them without any loss in image quality. But be careful: test on small samples first and learn their specialties and quirks.

An alternative to running them locally is to use Yahoo! Smush.it, which is an online service that  “uses optimization techniques specific to image format to [losslessly] remove unnecessary bytes from image files” using the same tools. The best way to run Smush.it is via the Yahoo! YSlow for Firebug, an add-on for the Firebug add-on for Firefox. (By the way, Smush.it renames GIF images to .gif.png when it shrinks them. I wrote a console app to rename them back to .gif.   Browsers only look at the image header to identify images, so it’s safe to serve up PNG images with a .gif extension.)

Mac:

ImageOptim Screenshot
ImageOptim

For OS X, all you need is ImageOptim, which optimizes JPEG, PNG, and GIF auto-magically.  Seriously awesome tool.  (Free.)

For lossy optimization, JPEGmini is amazing.  It uses knowledge of human visual perception to shrink JPEG’s up to 5X without visible quality loss.  (Semi-free.)

Windows:

RIOT (Radical Image Optimization Tool):

Though it has a batch mode, this is the best tool for optimizing single images, whether they are JPG, PNG, or GIF. I use RIOT to save every image I work on as well as to reduce the size of existing images that are too large. You can re-compress PNG and GIF images losslessly, but for JPG you want to save from the original file.

RIOT is available as a standalone version as well as a plugin for several image editors such as the excellent IrfanView.

 

RIOT - Radical Image Optimization Tool screenshot
RIOT – Radical Image Optimization Tool screenshot

The JPEG Reducer:

Run this tool in bulk on all your JPEG images to save ~10% of the file size. This is a GUI front end for jpegtran, which optimizes JPEG images by doing removing metadata and other non-display data from JPEG images. Because it is lossless, it is safe to run on all your image. It will ignore any files you add which are not really JPEG images.

The JPEG reducer screenshot
The JPEG reducer

PNG Gauntlet:

This tool is a front end for PNGOUT which will losslessly reduce the size of PNG images. Warning: if you add GIF or JPEG images to it, it will create PNG versions of those images. Sometimes you want to do this, but if not, don’t add images to the queue.

PNG gauntlet screenshot
PNG gauntlet screenshot

Let me know if you have other tools or ideas for image optimization.

 

tips

Taking Screenshots in Mac OS X

1 Comment

MacRumors has pretty much everything you need to know.  You can also take screenshots remotely, over SSH.  (Just remember that with great power, comes great responsibility.) The only thing the built in functionality can’t do is take full screenshots of windows that scroll beyond the screen.  There’s tons of commercial apps to do that, but Papparazzi! will do the job for free.

tips

Change file associations in OS X

1 Comment

Do you want to open a particular file with another program in OS X?  If you want to open it just once, you can right/control-click and select “Open With.”  (Since I have VMware Fusion installed, I can open files in Windows directly from the Finder!)

To open a single file with a particular program every time, right/control-click, and Get Info.  Then you can select to program to use under “Open with.”

This change only applies to current document.  To change the default program for all documents, click “Change All…” and confirm.

tips, Uncategorized

Using the new ClearType fonts on your Mac

Leave a comment

One of the nice touches of OS X is the use of the Helvetica typeface for rich-text editing in TextEdit and other programs. Microsoft’s version of Helvetica is Arial, which is basically a lower-quality rip-off of the original. (To understand my attraction to Helvetica, I recommend watching Helvetica, the documentary.)

However, in Vista/Office 2007, Microsoft released the ClearType Font Collection, a great new set of fonts, including several designed especially for viewing text on computer screens. The two I find particularly useful in this regard are Cambria, which is optimized for viewing small text, and Consolas, which is a monospaced font useful in programming and the like.

So how do you get these fonts in your Mac? Well, they come with Office 2008 for the Mac. After you install office, just go into your favorite editor’s preferences and select Cambria and/or Consolas as your default font.

If you don’t have Office 2008, things are a little trickier. This tutorial will guide you through installing the fonts in Windows and Linux – and OS X, if you install the required Linux utilities via something like MacPorts. Office 2004 users can get some of the fonts with the Open XML Converter. If you have a Vista computer, you can copy them from C:WINDOWSFonts to /Library/Fonts. If you have a pre-Vista OS, you can get them with the free Powerpoint 2007 viewer. If you are still out of luck, you can always purchase them directly from the foundry.