Four Lessons Learned From An Excursion Into Solo Web Programming

An embarrassingly long time ago, I decided that as a "side project" I'd implement a website useful for keeping track of my time, and on the way I'd improve my understanding of a few technologies about which I was either completely clueless (AJAX) or pretty inexperienced (Ruby, Rails, CSS, and countless others). That "side project" has become an excellent example of the perils of featuritis, particularly when the programmer in question has both too much time on his hands and a fetish for overgeneralization. It grew to suck up almost all my programming time and is now a user-programmable Facebook application for tracking, graphing, and publishing just how drunk you are (in addition to countless other things). The resulting website, Procrastaway, is probably something only its mother could love, but a few valuable lessons were learned in its construction.

I'm not going to claim that any of these lessons are terribly profound, and most experienced software engineers have probably heard most of them at least once before (or figured them out for themselves). Some of them may actually be half-truths at best; I'm no genius, and have undoubtedly spread myself a little thin in the process of creating the entire site on my own. Still, if this post is the one extra little bit of repetition that manages to save some other poor slob from making just one of the same mistakes I've made, it will have been worth it. Plus, I can always use the extra practice communicating via actual language rather than computer code and grunts.

Lesson #1: Facebook Sucks (Or: Fucking Facebook Fuck Fuck Fuckity-Fuck AAAAARRRRRRGGGGGGHHH)

In my first draft of this particular lesson, I tried to keep the discourse reasonably civil. Really, I did. I think I even mostly succeeded. But then, shortly after finishing the first draft of this entire blog post, I figured it was finally time to add my own app as a tab on my own personal Facebook profile. At long last, I'd show off the results of all my hard work to my family and friends! Triumphant, I moved my mouse over to the "Add Profile Tab" button on my Facebook app's main page.

Click.

"Wait, what? This worked on my test accounts just a week or two ago. Well, maybe Facebook's servers are just too busy shitting their pants right now to service the request. You know, like they usually are. Whatever. I guess I'll keep my fingers crossed and try again later."

I tried again later. And again later. I tried several times over the span of a few days. Each time, I'd get the same error message. And the error message would come quickly. Too quickly.

Uh oh.

"Surely they can't have deliberately broken this, can they? After all, just three or four months ago they were telling me that profile tabs were going to be the only way for apps to show up directly in people's profiles (not including their News Feeds), and they'd be throwing out the other, older ways Real Soon Now. They can't possibly have changed their minds and completely disabled those profile tabs already, right? Plus, the 'Add Profile Tab' button is still sitting right there on my app's page, rendered by Facebook itself, and there's no note in the online documentation for '<fb:add-profile-tab/>' that it's been deprecated. And what sort of blithering idiots would permanently disable the functionality but leave the button sitting there just so it could barf on you and tell you 'Please try again later'? Oh, wait, that's right, I already know exactly what sort of blithering idiots would do that: Facebook's blithering idiots."

Bracing myself for the worst, I searched Facebook's developer forum for "add profile tab".

Sure enough, it turns out they decided three months ago that they were completely getting rid of user profile tabs Real Soon Now (on top of the other profile integration techniques they'd previously decided to trash), announced the decision on their goddamn blog, and then completely failed to update the online documentation at all for three months before abruptly breaking the functionality in one of the worst ways possible.

Yep. Fucked over by Facebook. Yet again.

Facebook knows it's the only real social networking game in town right now, and will almost certainly stay that way for the foreseeable future thanks to the magic of network effects no matter how badly it pisses everyone off. So, it's going to continue flailing around with its user interface using Facebook's patented "random walk" UI optimization strategy, and it's going to drag its APIs and all its third-party developers along with it. Third-party developers have precisely two options: bend over and squeal like a pig, or kiss off.

If you ever consider creating any sort of Facebook app, know this: a Facebook app is not something you can ever "finish" and then just be happy with. Since I started playing around with Facebook development about a year and half ago, Facebook has deprecated, broken, and/or completely dumped every single one of their APIs that I have ever used. I am not exaggerating. I honestly can't think of a single counterexample.

The moment you start developing for Facebook, you become Facebook's bitch. Either you read their developer's blog slavishly to keep up with the arbitrary developer-hostile changes that get announced every time Mark Zuckerberg's butt itches, or you decide "screw that, I have better things to do", in which case your app will probably be broken yet again within a few weeks. At that point, you get to decide whether to (A) try debugging it and/or grovelling through Facebook's so-called "documentation" to figure out what just happened, or (B) just give up and let your app die an ignominious death.

This is all assuming, of course, that your app is actually broken, and the exciting new glitches you're seeing at any particular time aren't just a consequence of the fact that Facebook's servers are about as speedy and stable as a horde of drunken hookers on unicycles. That's right: once you build a Facebook app, you have now entangled your own system with another system over which you have absolutely zero control, and which is embarrassingly slow and explody. Not only is the Facebook half of this equation a pain in the ass to deal with in its own right, but it also makes testing and debugging the pieces of the system you do control that much harder. Suddenly you notice that a piece of your site that worked a week ago now hangs or otherwise behaves strangely; all right, was it something you changed in the meantime, or is some Facebook server occupied with picking its teeth off the asphalt?

Since many applications do need access to the social graph in order to have a reasonable chance of survival, and Facebook now owns the social graph, it may make perfect sense for you to go ahead and put up with this horseshit. If you've got enough people on your team that you can afford to have one person spend about half their time boaslapping and getting used to it, it might not even be excruciatingly painful in the long run. On the other hand, if you're a solo developer, you should be especially wary of getting involved in anything with that much churn.

Lesson #2: Solo Software Development Mostly Sucks

Working by yourself, particularly at home, is both awesome and horrible regardless of what it is you're working on. There are some important things to keep in mind about working as a solo software developer in particular, though.

In the last year or two of my previous job at a large software company, I probably spent 20-30% of my time on the "software engineering treadmill": namely, coping with sudden changes to the underlying codebase I was using that had to be dealt with before I could make any real progress. These changes were usually made by engineers on a completely different team, without any input from or warning to me. Given the circumstances, this was to be expected; everything was on the bleeding edge, and the stuff the other team was developing was more important to the company and to the world at large than my stuff. The treadmill was, however, definitely going to be one of major things I wouldn't miss after leaving the company and striking out on my own.

The good news about going solo in your software development is that, yes, you can step off the software engineering treadmill — to some extent. Nobody's going to yank your own codebase out from beneath your feet. You get much more freedom to choose when to upgrade the libraries, compilers, etc. that you're using, although you might be surprised how often practical considerations can still force your hand there. The bad news is that technology will merrily continue marching along outside your little time-frozen bubble, and when you do need to catch up, there won't be anyone to help you do it.

If you decide your project needs to integrate, with, say, Facebook, and then Facebook tweaks or completely scraps the APIs you've used, there will be exactly one person capable of hurriedly picking up the pieces: you, the same person already responsible for everything else. Furthermore, even if you do manage to keep your project from depending on any technologies that are going to change too quickly, there's a very real danger that by the time the project is finished, it'll be too late. Having only one person on the team means not only are you the sole engine of progress for the project, but there's nobody else around to give you a nudge right away if you start slacking off. Assuming you're doing something worthwhile, chances are high that competitors with multiple developers in their teams will show up with complete or near-complete products while you're still in the middle of developing yours. Good luck keeping up with them, or frantically trying to retarget your product for a slightly different (and in all likelihood significantly less lucrative) market. And by the time you're done with your product, it may be technologically outdated regardless of any competition it might or might not have. Wait, why is my product targeted towards desktop browsers when it's precisely the kind of thing that people would rather use on their smartphones? Oh, that's right: smartphones barely even existed when I started it. Why does my system do so much work on servers I have to pay for when I could be doing it in the client's browser for free?! Oh, that's right: three years ago, browsers couldn't really do what I needed them to. (And, hell, IE still probably can't.)

Another major disadvantage to working on software by yourself: sometimes, it's tempting to take the quick and easy way around an annoying technical issue that really needs to be properly addressed head-on before anything else is done. Now, nobody is around to give you the requisite Look of Disapproval when you say thinks like, "Putting all these parameters into their own table and making sure the database is still normalized would just be so gosh darn inconvenient. Eh, let's just dump them all into a text column as a JSON-encoded hash and be done with it." There may be nobody to help you brainstorm a decent solution to this or any other problem, either (unless you swallow your pride and go bug total strangers on, say, stackoverflow, which might work). And most importantly, nobody's going to be around a few months later when you'll really wish you had a good scapegoat to blame for that dumbass JSON column.

On the positive side, nobody's there to look at you funny when you set a dual-monitor workstation up on top of a TrekDesk [no affiliation], buy a literal software engineering treadmill, and exercise while you code. I've found this actually works pretty well...if I set the speed at a leisurely 1.3 miles per hour but max out the incline, I can burn (if the treadmill is to be believed) about 1000 calories in three hours while working before I've had enough walking for the day. At that speed, typing and reading is still pretty easy for me. I do work up a sweat with the incline maxed out, but, hey, nobody's there to complain.

Plus, nobody's going to tell you "no" if you decide it'd be amusing to blow a couple of days trying to make your Terms of Service as humorous as possible while having them remain both legally binding and usefully ass-covering. (Truth be told, I'm probably prouder of those Terms of Service than most of the rest of the site.)

Then, of course, there's the all-important fact that you're not paying anyone else any actual money for development. And, hey, if you pretend your own time is completely worthless — or somehow manage to convince yourself that you're having sufficient "fun" while "working" — then that means the entire project is effectively being done "for free"! Yaaaay!

Lesson #3: Users Suck (or: UI Design Is Important)

However hard you, the programmer, might think that your users will suck, they will almost assuredly suck harder. I mean this in a specific, totally non-judgmental way. When presented with a complicated website (or any other software artifact with a user interface), they will attempt to navigate it using only the lizard parts of their brains, and if that fails, they'll give up and go back to surfing for porn in ten seconds flat. Not because they are inherently stupider or lazier than you, the architect of the greatest pieces of software since sliced ether, but because they're an awful lot like you (but do not == you).

Namely, much like you, they've got better things to do than try to read your frickin' mind. Even if you spell it out for them. If you think your website is going to be so compelling that they're actually going to read anything approaching "documentation", you are almost certainly letting your ego blind you to reality. Never mind documentation sitting "off to the side" on a whole separate page; even if you write just a sentence or two of documentation and place it right next to the goddamn user interface control it's talking about, the user will most likely completely ignore it, and then get annoyed when things don't happen as they expected. I've seen this happen in my own user testing, even when the user in question was simultaneously (1) literally a rocket scientist, and (2) my own wife — which implies, among other things, that this particular person normally has a stupendous amount of patience. And when I carefully watch myself interacting with other people's websites or software — and put my ego on hold — I see myself doing the same thing. Anything more complicated than an immediately recognizable icon or a three-word phrase is immediately translated by the web-surfing lizard brain into "blah blah blah", and may as well be in Swedish. (Someone should create an "Encheferizer" user interface design tool that helps point out bad design by automatically replacing all text phrases of four or more words in the interface with "Bork bork bork".)

This is undoubtedly old news to anyone who's done any serious user interface design before, or anyone who's read Don't Make Me Think (recommended). I, unfortunately, had been much more of a backend C++-type programmer, and naively figured I'd just make a hugely complicated system that did stuff I wanted it to and then figure out the actual user interface later. Do not do this, particularly if you are lazy (like most good programmers) and/or dislike user interface design (like most non-UI specialists, I suspect). If you do, it will be all too tempting to allow the underlying architecture of your system to dictate the user interface for you, and that interface will then probably be near-impossible for users to understand. Creating a sane user interface for a complicated system is hard to begin with; creating one using abstractions that don't match up with the abstractions of the underlying system because those underlying abstractions are inherently user-hostile is much harder still. If you're designing a system for hapless meatbags to interact with primarily via clicky-clicky, design the user interface first whenever possible. Seriously.

Moreover, if you're not sure you can make a reasonably simple interface for the system you have in mind, you should give some serious thought as to whether that system is going to be worth building at all. However user-hostile Procrastaway's interface might be, it's an utter breeze to use compared to the ridiculous prototypes I'd been building for the other "main project" website I'd been working on before I got sidetracked with Procrastaway. After my first experiences with letting actual users try out Procrastaway, I quickly realized that my previous "main project" was in fact fundamentally hopeless: not only was the user interface I'd built for it unusable, but any user interface I could possibly build for the system I had in mind was going to be too complicated to win over a reasonable number of users. A true UI expert might conceivably find a way to build one, but I'm not that expert, and I'm not actually convinced anyone else would be able to build one either. So, I killed the project, and another little piece of my naive "Everything is possible with computers!" programmer's optimism died with it. Oh, well...good riddance.

Of course, if you're going to worry about your user interface first, then the very first thing you need to know is what your user interface options are. Which brings us to:

Lesson #4: Web Browsers Suck (or: UI Design In HTML is Impossible)

OK, maybe not totally impossible, but sweet Jesus is it ever close. HTML by itself isn't too bad. JavaScript by itself isn't too bad. But mix them together, sprinkle in some CSS, and then top it off with a big steaming shitheap of browser bugs and incompatibilities, and it's enough to make a programmer lose his will to live if he tries anything even remotely "interesting" from a UI perspective, particularly if he's not already an expert in front-end web programming.

HTML was, of course, never initially designed to handle any sort of real user interface. The authors of Ye Olden HTML Documentse weren't supposed to care how their documents were formatted beyond "Here's a header. Here's a paragraph. Um...whatever. Render it however you like, Mr. Browser." Then of course it turned out that, no, a lot of people do care very much what their documents look like, so CSS was kludged in as an afterthought — and boy, does it ever show. As a way of changing how a particular single piece of text is styled, CSS is probably OK, assuming you start off with a CSS reset that minimizes the differences in the default rendering styles across browsers. (I didn't start off with a reset, and by the time I figured out that would have been a good idea, retrofitting things looked like it was going to be too painful.) As a way of determining how blocks of text are arranged on a page, though, it's a nightmare.

Even simple layouts — say a couple of columns for your main text, a header, and a footer — are surprisingly hard to get right, and if you try to do anything more complicated than that purely with CSS, you're in for a world of confusion and hurt. (You know some unfortunate technology decisions have been made up the line when you find yourself thinking "This would be so much easier if I could just use Java's GridBagLayout instead.") Many CSS experts will tell you that you really really need to "separate content from presentation", and will act as if you're faintly retarded if you fall back on using tables for your formatting instead, since Tables Are For Tabular Content. Everyone should ignore those sanctimonious pricks and feel absolutely no shame about giving up and using tables. Sure, in theory, separating content from presentation is a good idea. In practice, using CSS for layout is confusing, error-prone, and brittle, and that's before you even take browser bugs and incompatibilities into account.

For example: after a large amount of time and frustration, I did eventually settle on a CSS-based solution for positioning the footer in my site layout, since I wanted the footer fixed to the bottom of the browser window, and you can't do that with tables. It mostly worked, but was completely broken when viewed on iPhones, so I gave up and special-cased a footerless layout for iPhones. (I'm not sure why I bothered, given that the layout on the iPhone is broken in so many other entertaining ways that I'll probably never going to get around to fixing.) I'd feel worse about the footer not actually working across all the browsers I cared about, but the similar footer on Facebook's site was broken in exactly the same way on the iPhone, at least before they removed the footer from their site entirely. That's right: even the experts couldn't make a fixed CSS footer that really worked. (Or at least one would hope the HTML/CSS designers at Facebook would count as "experts"; then again, see Lesson #1.)

And that's all assuming you're just trying to get a boring static page to look nice. Want to try implementing an even moderately complicated drag-and-drop user interface? As far as I can tell, unless you enjoy the thought of dedicating your life to learning all the intricate ways in which various browsers are broken, you'd be better off just shooting yourself in the head. Yes, there are JavaScript libraries dedicated to making drag-and-drop operations possible. Yes, they'll work nicely in simple cases, thanks to all the browser-specific hacks they've got coded up inside them. But try anything out of the ordinary, and they'll fail miserably. The hacks in these libraries don't fix the fundamental truth that HTML, CSS, JavaScript, and browser bugs have all coalesced to form a horrendous eldritch Non-Orthogonal What-The-Fuck Beast bent on sucking your brain out through a crazy straw: they merely hide it. And Lord help you if you try anything complicated enough to expose it again. Sluuurrrrrrp.

Example: I wanted an interface in which users could arrange items on the page into columns, which would in turn be arranged in rows. Sounds like a fine application for drag-and-drop, right? You'd just have nested drag-and-drop areas where items could be rearranged within a column, columns could be rearranged within rows, and rows could be reordered. No problem. How hard could it be? Well, after a few days of futzing around with the drag-and-drop bits of scriptaculous (the JavaScript interface library popular with the Ruby on Rails community at the time) and stomping out annoying corner cases, I finally had something that worked in some barely-tolerable fashion. Yay! Or thought I did, anyhow, until I tried it in IE7. (You saw that coming, right? And no, I wasn't dumb enough to even pretend my site might ever work in IE6.) Pieces of my user interface were hiding behind other pieces. No amount of flailing around with z-indices made the problem go away. If I'd had perfect knowledge of IE's rendering bugs and a superhuman supply of patience, I'm assuming I could have figured out a way to make it work. As it stood, I gave up and designed a completely different interface with no drag-and-drop that is probably faster than drag-and-drop for "experts" (i.e., me) to use, but "run away screaming"-level intimidating to everyone else.

And while web developers like to swear at Internet Explorer for being the source of most of their browser-related woes — for good reason — the truth is that all browsers I've tested my site with have had pretty disgusting bugs. I fully understand that modern browsers are incredibly complicated pieces of software, and it's sort of amazing that they work at all; if it were me implementing them, chances are they'd be much worse. This is, however, of little comfort when one is presented with a show-stopping bug with no discernible reasonable workaround. Firefox bit me by occasionally and nondeterministically executing "onload" code before the rest of the page had actually loaded; Google searches suggested this was some sort of caching-related bug. Solution: painstakingly adding checks to my onload routines to make sure that everything they needed was already defined, and scheduling themselves to get called again in a fraction of a second otherwise. Safari bit me by adding extraneous formatting gibberish to the contents of text areas whenever the user pasted text into them; Google searches suggested this was a Known Issue. Solution: "Well, I guess in this case it'll just suck to be a Safari user."

I suppose this is where I should try wrapping up what has really been more of an extended whinge into something more closely resembling an actionable "lesson". So here goes:

(1) If you have good front-end web programmer/designers, you probably aren't paying them enough. Imagine being a highly educated garbage man, except about one third of the smelly garbage cans you encounter abruptly try to kill you, and can only be defeated by frantically (but precisely!) shouting garbage-can-dependent arcane incantations like:

.quux {
width: 500px;
padding: 50px;
voice-family: "\"}\"";
voice-family: inherit;
width: 400px;
}
html>body .quux{
width: 400px;
}
Ia! Ia! Tantek Fhtagn!

Sounds like fun, no? No. Everybody should be nice to these people.

(2) Think very hard about whether you really want to design any reasonably complex user interface in HTML, CSS, and JavaScript instead of either (A) programming a native application or (B) using some other browser-based user interface framework like Flash (or perhaps coding the entire interface in an HTML 5 canvas if you're some sort of "standards" fetishist). Not that I've actually tried doing anything in Flash or HTML 5 canvases, but seriously [cue: Famous Last Words], how could anything possibly be more painful than designing user interfaces in HTML, CSS, and JavaScript?


That's all for now. Be sure to tune in again two years from now for our next thrilling blog post, "Five Lessons Learned From An Excursion Into Mobile Phone Game Programming." (Lesson #1: Fucking Steve Jobs Fuck Fuck Fuckity-Fuck Arrrrrrgh.)



Wait a minute. You never even read down this far, did you? You took one look at how long this blog post is, and right now you're actually back to surfing for porn, aren't you? Aren't you?! See the URL of this post? Yeah, that's right, I already cut the number of lessons down from five to four because I knew I was too close to the edge of your gnatlike goddamn attention spans. But that still wasn't enough, was it? Nooooooo, of course it wasn't. Even after all this time, I still haven't truly absorbed Lesson #3. I clearly should've made it Lesson #1 instead. Christ on a bike, you users suck. Good thing you're not actually reading this fucking paragraph.