Double Digg Destruction Aftermath

October 31st, 2006

Webomatica is still recovering from the wave of double digg destruction that swept in last weekend. It started with my first ever digg front page story, Seemingly Stupid Apple Moves That Were Actually Brilliant. The resulting “digg effect” took my site down. I then wrote another blog post Digg Destroys Webomatica, which also hit the digg front page, ironically destroying my site once again. This all happened over three days.

Anyhow, my site is now back up and I’m sorting through the pieces. I’m now able to present some factoids that may prove interesting.

Some Numbers:

Seemingly Stupid Apple Moves That Were Actually Brilliant: 866 diggs, 12,091 page views.

Digg Destroys Webomatica: 805 diggs, 6,343 page views.

Alexa: This chart is pretty amusing, basically a vertical line going straight up. This demonstrates both how low my traffic was before and the sheer number of visitors that suddenly arrived. It’s also crazy to think that this line could have gone higher if my site hadn’t choked (twice).

The realist in me is certain that this will be a vertical line going straight down in a few days. But it’s pretty amazing to see that for one brief moment, my site had an Alexa traffic rank of 5,744.

Technorati: I gained a few blog links, so my Technorati rank edged up slightly to 105,606. This is based on added links which is more permanent than traffic and I’m happy about this. The all-knowing blog worth evaluator now coughs up $14,678.86. I wonder how many burritos I can buy?

Side Effects

My other site, Bottle Cap-O-Rama (showcasing my bottle cap collection) also saw a huge traffic increase. I have a link to it in this blog’s sidebar and footer.

I was also pleased to see that a fair number of diggers actually checked out other blog posts. Although the vast majority glanced at the dugg articles and left, it wasn’t everybody by a long shot.

Tech Support

I received some technical suggestions on ways to prevent another death by digging. They seem to center around two issues: WordPress and hosting.

I learned that my installation of WordPress is not efficient with high traffic. I’m not alone; it’s common to see WordPress blogs choke under the digg effect. I’m implementing these WordPress tweaks:

1. Adjust my theme to use less images.

2. Optimize the remaining images, crunching them down in the interest of smaller size over aesthetics.

3. Have the main theme images stored on another server (Coral cache or other), and write emergency header file that points to these images.

5. Rethink the use of all my plug-ins that are slowing down page loads (lightbox JS comes to mind. Also the hit counter).

6. Use the WP-Cache plug in (see below).

7. Look into Feedburner.

8. Look into a social bookmarking bar.

Now on to hosting. Here’s a list of suggestions from goatrandy, which I don’t completely understand, but I’m working on (slightly edited for readability, and I added the links):

1. Get a VPS hosting package. Make sure you ask any potential host what the maximum file handle limitation is for the kernel they use though. Many have this set very low, and if you can only have 1,000 files open at once than you’re doomed. You need at least 10,000.

2. The VPS will allow you to configure your server however you want it configured. This means you can use an alternative HTTP server like lighthttpd. My crappy VPS with only 128 megabytes of “virtual ram” could serve something like 10,000 static html pages per second with lighthttpd (LINK).

3. If it’s a WordPress blog, then ALWAYS use the wp-cache plugin.

4. Bandwidth is expensive, so if your article has many images or a binary download, change the links to use the Coral cache.

5. If all of that still isn’t enough, look into configuring a reverse proxy with Squid. This way, everything is cached as static content by squid running on port 80, and your webserver runs on 8080 or some other port. It will drastically reduce CPU, and memory requirements.

Note: What technical solutions do you suggest? I’m all ears at this point. Dealing with high traffic is not my area of expertise, but I’m willing to learn.

In conclusion, as with anything in life, you learn from what’s happened, try to rectify your mistakes, and move on. Although I’m enjoying the sudden attention and positive feedback, I’m still looking at this realistically, as a month from now, my traffic will likely have seeped to a trickle and the weekend my site was blown to bits by digg will be a distant memory. But along the way I got some positive comments about both the design and writing of this blog. I feel vindicated after wondering if anyone was reading my writing.

The next time I get a visit from digg, I hope to better prepared and more able to take advantage of the situation.

Please subscribe to the Webomatica RSS Feed!