sparkfun is down

Sparkfun's IT administrator laments about MySQL while site is down. Copied here because it'll be gone tomorrow.

tl;dr: SparkFun's down for a while this evening, starting at 18:30pm MDT, while we work on some database stuff. We should get better at uptime, but right now we're mostly trying to get better at databases. Related web nerd musings below the llama, for those interested.

So tonight, sparkfun.com's going to be down for a while. We're hoping for no more than an hour or two, although it could very well run longer.

This is where I concede that we could really get better at keeping the site up all the time than we are right now. We're working on that bit, but right now what I want to ramble about instead is databases.

Since approximately the dawn of time (which is to say since some time last decade, before I worked here), SparkFun has run the core of its business on MySQL (and recently MariaDB). When I got hired in the fall of 2007, a lot of the important code was still split between a hacked-uposCommerce installation and a pile of MS Access clients. That stuff has been gone for years now, but the database itself persists. In places it's practically a fossil record of offhand design decisions made by other people writing a shopping cart some time in 2003.

This probably happens everywhere. Application code may change drastically, but in a working shop, the data store has a way of becoming the scaffolding around which (for better or worse) everything is built and must be maintained.

As it happens, when we first realized that we needed a better toolset than not-very-object-oriented old-school PHP, we made some decisions with long-term consequences:

• We decided to keep using PHP instead of rewriting everything in Ruby.
• We decided to write our own lightweight web application framework instead of using existing PHP projects.
• We decided to write our own ORM layer instead of using existing libraries.

A lot of things informed those decisions: We didn't want to have to replace all the working software at once just to use better tooling for new code. Everyone on our team already knew PHP. The frameworks we looked at seemed lacking in comparison to the stuff we'd used in languages like Ruby and Perl. The available PHP ORM libraries were either heavier than we wanted or built on assumptions that didn't fit our existing (frequently terrible) schema. Finally, rolling our own sounded kind of interesting, and we had no idea what we were getting ourselves into.

(Ok, so in the interest of full disclosure, what happened is that I talked louder than people who knew what we were getting ourselves into.)

Fast-forward a couple of years during which we hire more programmers and SparkFun gets another half million orders. We're sitting around in the mountains drinking beer and talking about what we really should get better at and the subject of PostgreSQL comes up.

See, MySQL and its forks, for all of the really huge and amazing stuff built on them in the last decade, have got some drawbacks. On this point, I risk triggering a lot of strong feelings from the sort of people who have strong feelings about databases, so for the time being, let's just say that it started to seem like we'd have a better time writing business software on a database that embodied a more rigorous approach to formal types and relationships. And because of years of our nerd friends telling us we should probably just use PostgreSQL, it kind of seemed like PostgreSQL might be that database, especially since we already knew we were going to use something with open code.

So we had some more beers and we talked about it and we had some more beers and we talked about it some more, and maybe six months later we finally decided to go for it.

So that's what we're doing.

Sort of.

It turns out that, when you write your own ad hoc ORM layer against a piece of software like MySQL without the faintest idea of how far you're about to be in over your head, it's pretty easy to make a bunch of assumptions without even noticing them. Rumsfeldian unknown-unknowns, if you will, creep in at every turn. Like fish and the concept of water, people who work with a given data store for a long time begin to forget that certain abstractions are even objects of possible consideration. These bits of data are represented as strings in PHP? Well, that's natural, right? What else would they be? String comparisons in your database of choice are all case-insensitive? Well, heck, that's almost a feature, right?

So then you decide you're going to point things at a different database and you get a crash course in all of the stuff you were wrong about, and all of the things you made impossible to test, and all of the ways your implicit assumptions about the two or three or five different type systems you rely on across languages and representations of your model are actually pretty fractured and subtly wrong.

So what we're doing tonight is rearranging a bunch of MySQL schema and deploying a terrifyingly long list of code changes that get us about three quarters of the way to being able to actually migrate to PostgreSQL without all of our software exploding continuously. We sincerely hope. (As regards exploding software, I also have a lot of recently developed thoughts about how I'll never again write a large project in a dynamic language without just tons and tons of unit tests.)

I should really write something coherent about all of this, but I just realized we actually need to deploy the code and such now.

Anyway, thanks for bearing with us while the site's down. The colored wires and blinkenlights and such will be for sale again soon.

☮,

— Brennen / SparkFun IT

postscript: This is the best sloth, right?

The State of Mobile VOIP: Redphone

A couple years ago, Whisper Systems released a product called Redphone that is quite intriguing. Redphone is both an app for Android and iPhone, as well as a service (like Skype). The easiest way to describe Redphone is that it's SIP with a fixed configuration, ZRTP and TURN. That is:

• all the parameters are set by the system making configuration trouble-free
• all the calls are encrypted with ZRTP
• while SIP is usually peer-to-peer, Redphone calls go through Redphone's servers.

You register with Redphone using your phone's number. (It also works with Google Voice.) Redphone verifies it with a text message that it automatically responds to.

When you call, you specify another phone number. You can use the stock dialler and, if the other person is registered, it will use that instead. Redphone "rings" this device using either an SMS message, or a Google or Apple push notification, whichever the receiver prefers.

NOTE: In my experience, Google push notifications don't appear to work (or at least not be reliable) over a NAT'd local network (most wifi). If you aren't receiving calls, try switching to "always use SMS".

If the call is accepted, the two phones begin an elaborate protocol to both encrypt the conversation and make sure that no one has intercepted the call in between the two phones.

After a call has begun, Redphone will display two words on the phone's screens. You read one word; the other party reads the other word. If the pair matches what the phones show on the screens, then you can be reasonably sure that no one is in between you two, relaying the call's data.

ZRTP specifies that, once you have called someone, you shouldn't have to go through the "compare words" protocol again, however rumor has it that Redphone doesn't store this information. I can't say I mind, since it seems (to me) like the best place to attack the protocol, and the word pair protocol is quite unobtrusive.

Redphone is fairly hassle-free. ZRTP doesn't require previously trading public keys with anyone. The security relies on you recognizing the voice at the other end, and the word pair. Calling someone is as simple as selecting them out of your contact list. If you or your communicado's are tech-newbs, but you want security, this is the app to use.

Redphone was written by Moxie Marlinspike, a name well known in the security business, and Stuart Anderson. Redphone was acquired in 2011 by Twitter, who seems to be trying to make a name for itself among Middle Eastern protesters. However, the service can and has gone away before. Have a back-up.

The State of Mobile VOIP: Republic Wireless

A company called Republic Wireless made the headlines a while back. They advertise that they will send a call to either wifi or the cell network, whichever is cheaper and available. This is basically the Holy Grail of mobile communication. As promising as this is, there are gotchas.

For one, Republic Wireless basically licenses coverage from Sprint not unlike Metro PCS and others. This means that your back-up coverage is fairly spotty.

Second, they want $20 a month. This is cheaper than a cell phone account, but only half what Metro PCS wants including data. I have a suspicion that you're paying for a "data-only" account.

Finally, you have to use their phone (a Motorola Defy XT). Even though the software would work fine on any phone, and even though there are other Sprint phones, they want you to use a specific (rapidly aging) model.

If you put this all together, Republic Wireless is not exactly the "liberator" that it claims to be. In fact, they are fairly rigid. It would be nice if one could use their own phone on their own provider, even if it's more expensive. Given all this, it may be just as cheap and convenient to track down a data-only provider, and just use SIP all the time.