Pinboard Blog

Technical Underpinnings

Some people have expressed interest in the technology that runs the site.

We have two servers, a dedicated machine from ServerBeach and a Xen instance from prgmr.com. Each of these has 4 GB of RAM. The site originally ran on Slicehost and Linode, but we moved off those services since their cost (expressed in dollars per GB of RAM or storage) was far higher, without any offsetting benefits.

Pinboard uses a master-master MySQL architecture. One of the masters performs all writes to the bookmark, user, and tag tables, while the other is used to calculate aggregates (like global link counts and per-user statistics). We also run our nightly DB backups off of this second server with mysqldump, storing the compressed results to S3.

A small batallion of Perl scripts handles background tasks like downloading outside feeds, caching pages for users with the archive feature enabled, handling incoming email, generating tag clouds, and running backups. I chose Perl for this partly because I'm productive in the language, and partly because CPAN offers usable libraries for any conceivable background task.

The site itself is built in PHP, without any templating engine. We use APC to prevent re-loading various files with each request, but have so far not needed to implement any other kind of caching.

The bookmark archive makes heavy use of wget, with some postprocessing to handle the common case where stylesheets and other inline content are versioned by appending a random string to the filename ("main.css?90xs8"). This versioning messes up Apache's ability to guess the content type of the file from its extension when it comes time to serve the archived file, so we do some renaming and symlinking behind the scenes to make sure archived files get served back properly.

We use Sphinx for our search engine, as well as for global tag pages.

Our technical goals are to never lose data, be very fast, and favor boring and faded technologies where possible. A rule of thumb that has worked well for me is that if I'm excited to play around with something, it probably doesn't belong in production.

—maciej on January 12, 2010