Desperate Measures - A Time To Hack

A Case Study

Sometimes the features that seem simplest on the surface are the hardest to implement. This statement rang true for me in the past few weeks. In today's post I will walk you through the stages of my latest feature request and the challenges I encountered. This example is far from my typical experience, but is an extreme example that illustrates the kind of problem solving and outside-the-box thinking required of professional software developers.

For a bit of background, my work involves payment card processing in various forms and across a wide range of systems and third-party applications. One of my most successful projects to date has been a full payment solution integrated with a third-party point of sale system. This new feature was an enhancement to that system and required customization of receipt printing as requested by a client.

In a typical setup, the system will print three receipts for each credit card payment: a merchant copy, a customer copy, and an itemized receipt listing all of the items and totals ordered by the customer. Our client must be environmentally friendly (or, perhaps, simply cost-conscious) because they asked to eliminate as much receipt waste as possible. When configured for this new feature, the system would print nothing by default. Instead, it would cache the receipt details for later printing if requested. "Hey, could I get a copy of my receipt?"

At first glance, this feature sounded pretty simple. Creating a cache to store data for later retrieval is the bread and butter of a developer's work with many tried and true solutions. I would have to research the point of sale system scripting language and communications protocols to install a new button that would fetch the receipt details from our cache and print the receipt on demand. This, I figured, would be the challenging part of the project. While challenging in its own right, this paled in comparison to one challenge that I had overlooked.

The overlooked piece was the itemized receipt. These details are normally printed automatically by the third-party system. "No matter", I thought, "there is an API call to access these details". I had used it in a couple of different places previously. It wasn't until I went to implement the API call that I discovered the problem. The API did not work when a payment was in progress or after the check had been closed on the system. This meant that by the time my software got involved with the payment, it was already too late to retrieve the itemized receipt details.

Desperate times call for desperate measures. We contacted the vendor to discuss possible workarounds. After some back and forth discussion, we learned that the system has a print-to-file receipt option. I figured this would be a good option. My software could monitor the receipt file, waiting for new data to be written. I could then parse and cache the latest receipt details for retrieval. I spent some time restructuring the code to monitor a file for updates and scrape out the itemized receipt details.

After struggling to learn how to monitor a file for changes, all went well with my initial test runs. The receipt text was written to the file. My software read the new data and cached it. Hitting the receipt button triggered the system to print out all of the required receipt details. I figured it was just a matter of tidying the code and running it through a battery of tests. I sat back and admired my clever solution to the problem, proud of implementing a reasonably complex solution to my seemingly simple problem. It wasn't until we brought a second machine into the mix that signs of a flaw in my plan appeared.

The point of sale system runs with a central server and multiple client workstations. I had performed the majority of my development work on the server for simplicity. On the server, everything was working seamlessly. On the workstations, we saw random delays from the time of processing to the time the file was actually modified. Sometimes it would take a few seconds. On other tests, the delay climbed nearly to the 1-minute mark. Waiting a whole minute for a receipt simply wouldn't fly with a busy merchant.

Further discussion with our vendor contact led us to conclude that this was simply the nature of the system. The file-based printing was designed as a record-keeping system, not a live processing environment. It was time to go back to the drawing board. Something about the best laid plans...

While discussing the problem with a colleague, I joked about it saying, "maybe we should just write a printer driver." My colleague took my joke a bit more seriously than I had expected and started playing around with the printing configurations on our test lab. He found a printer option identified as TCP/IP printer and pointed it at our software's listen port. We ran a test transaction and saw that some unknown request data had been logged by our software.

Somewhat bewildered and whimsically bemused at what I was about to try, I visited everyone's favorite search engine and started looking around for TCP/IP printer message format specifications. With a bit of trial and error and some luck, I stumbled upon a specification that looked similar to the messages that my application was receiving. "This might just work...", I thought.

According to the specification, the first message I saw was a printer status message. I updated my software to respond with a "ready" response. When I received a second request following my response, I knew we were onto something. I worked through the commands one by one until I saw the first line of the receipt header in our logs - plain as day.

There's something about this particular hack that is both wonderful and horrifying at the same time. I get a thrill out of the fact that we are simulating a printer simply to retrieve the information we need. At the same time, I boggle at the fact that something this simple shouldn't have to be so hard. There is also, unfortunately, a nagging risk that my software will be incompatible with some requests sent to it in the field. We will do our best to reduce this risk, but doing something that's not officially sanctioned and ordained by a vendor is generally a bad idea. Whatever way you slice it, this solution works!

Perhaps my favorite part of this solution is how I imagine the message exchange to determine the printer status:

THIRD PARTY:  Do you have paper?
MY SOFTWARE:  I have a roll long enough to span the country.

THIRD PARTY:  Is the paper door closed?
MY SOFTWARE:  It's glued shut.

THIRD PARTY:  Do you have ink?
MY SOFTWARE:  A well of ink so deep you could swim in it.

THIRD PARTY:  Is your cutter enabled?
MY SOFTWARE:  The blade is sharp enough to cut diamonds.

Hacks

Why do I have two simultaneous and opposing reactions to this hack? Why would some developers turn their nose up at my solution, while others would describe it by saying, "that's brilliant"?

The software industry has a long tradition of describing both clever and ugly solutions as "hacks". This is, I believe, because most programmers are very calculating people. We tend to have strong feelings about the "correct" solution to a problem. When something deviates drastically from that ideal, we describe it as a hack. At the same time, when there is no correct solution to a problem, the curious among us try to come up with a "clever" solution to work around the barriers set in front of us. We will often describe this as a hack as it deviates from the theoretical ideal solution in the same way.

Whether or not you like what I've done in the case study above, it's hard to argue with results. If an ugly hack is what stands between myself and my goals, I'll choose the hack every time (barring the ethically questionable). When there is no correct way to do things and I can find a workaround, you can bet that I will take the workaround. On the other hand, if there is a correct way to do things that cannot be accomplished under some immediate time pressure, you can be sure that I will want to come back and set things right when I get the chance. Remember, the customer is always right. The client doesn't care about your academic ideals -- they want results.

Quality developers usually need to think inside the box. The best developers will look at standard approaches to their problems. They will use proven technologies and techniques to get things done in a way that is robust and maintainable for the future. The best developers, however, have the ability to think outside the box when necessary. The resourcefulness and ability to bend a software system to your will is what separates the great from the merely good.

What is craziest hack you have seen or implemented? Tell me your story in the comments.

Cheers,

Joshua Ganes

My Love / Hate Relationship With Stack Overflow

From Day One

I was an avid reader of Jeff Attwood's Coding Horror blog from the time I left university and ventured into the world of professional software development in 2006. I had just recently discovered the ever insightful Joel On Software by Joel Spolsky when the two bloggers announced a joint venture to create a new resource for programmers to collaborate using a question-and-answer format. Providing a desperately needed service backed by the talent, reputation, and influence of two widely known developers, I had great confidence that this venture would be a success. I am proud to say that while I had nothing to do with the actual coding or design of Stack Overflow, at very least I had my say on the name. I participated in this poll and counted my vote among the 1721 supporters of the eventual winner: stackoverflow.com.

Falling In Love

Perhaps my favorite thing about Stack Overflow (and the family of Stack Exchange sites) is when I try to ask a question that has already been answered by the community. I can quickly find similar questions to my own with all of the answers ranked based on how helpful the community found them. I can't tell you how many times I've paid a visit to the Stack, only to find that my question was already answered clearly enough to proceed immediately with my work.

In the rare case where I have not found the information I was looking for, I have been surprised by just how eager the community was to leap to my rescue. Helpful answers began to flow within minutes of posing my question. Intelligent developers from around the world competed with each other to offer helpful answers to my question in return for nothing more than my gratitude and some (nearly) meaningless reputation points.

The Hatred Begins

There was once a time where open-ended and opinionated questions would be asked and discussed ad nauseam. These questions would divide the community into cliques, each supporting their own point of view and uniting in defiance of all who would disagree. I, personally, found many of these discussions fascinating. It was exciting to see the varying perspectives and how viciously they were defended by their champions.

While some (like myself) loved these types of discussion and debate questions, others wanted these questions to fade away. The most adamant ones wanted them dragged out into the street and shot. They didn't like having questions with multiple conflicting answers or no real answer at all. Eventually, word came down from on high that every question needs to have a real answer.

This requirement did not destroy the world (or the site), but it began to slowly erode part of the community and its nature. Developers, as one might expect, tend to be nerdy. Many of us are pedantic (guilty as charged) and love to enforce stringent rules to the letter of the law. Those who are most eager to do so often find their way to elevated positions of authority also known as moderators. I am under the impression that some of today's Stack Overflow moderators are on a quest to smite questions (be they useful or no) before they can be answered. They close questions as off topic or not constructive or a wide variety of other resolutions if they even hint at straying from the true and acceptable path.

The biggest offenders I've seen are of the form: "what is the best tool for X?" These are usually perfectly sensible questions. I need to accomplish X. I've probably done some searching and found a couple of tools that claim to help with X, but I have no experience with any of them. I attempt to consult the community for its collective experience with these or other tools in the quest for X. I find that these questions often have a couple of brief answers and have been ruthlessly closed as not constructive. The very nature of this type of question makes it subjective, but there is still a lot of valuable information to be shared here. Instead, the community is stifled by an overzealous moderator too eager to close the question to see the tremendous value that its discussion would bring to the site.

The other casualty of this is those active members of the community who lost interest following the change of course. Instead of participating in an exciting discussion with intellectual equals, they were challenged to answer technical questions with correct answers. I guarantee you that some of these members reduced their level of activity on the site as a result.

Sites vs. Tags

My next beef is with the Stack Exchange family of sites. As more and more groups begin to create Q&A communities on Stack Exchange, it becomes harder and harder to find the right site to pose my questions or look for answers. Depending on the nature of my search, there may be a fair number of relevant sites to look at.

Imagine that you run into a library issue while programming a GUI application primarily targeting Ubuntu Linux. Where should you go first? Ask Ubuntu? Unix & Linux? Programmers? Stack Overflow? User Experience? You might think that some of my suggestions are more probable than others. My question is: why should you have to think about it?

Rather than running many similar but separate sites and migrating questions between them, why not put it all in one place and tag each question as belonging to multiple relevant categories? Stack Exchange can still maintain communities centered around each category. The moderators can still callously and judiciously kick inadequate questions out of their domain. Why should I have to choose a site? Just let me ask my question and allow the community as a whole help direct it to the place where it belongs.

Crawling Back For More

Even with the problems I've discussed above, Stack Overflow is still a fantastic resource. When I have a programming question, I consistently find the best answers there -- even if I make my way there by searching Google. The fact of the matter is that with a large, intelligent, and active community, Stack Overflow is the best resource I know for all flavors of programming minutiae. I simply could not perform my job as efficiently without it. I guess I have to accept its flaws and admit that I still love it after all.

What are the best and worst features of Stack Overflow for you? Tell me your thoughts in the comments.

Cheers,

Joshua Ganes

Pulling The Plug - Software End-of-Life

Recently, the company I work for underwent a serious hardware overhaul -- upgrading a large swath of 32-bit servers to 64-bit processors and updating to the latest operating system and application versions. All in all, this upgrade went very smoothly. We planned out this transition and tested our software in our development environment before deploying to production. We ironed out a few issues that cropped up within the first few days. The problems ranged between minor incompatibilities and simple human error. Unfortunately, hiding in the shadows was a long-forgotten application that was still in use by a handful of clients.

This piece of software is the kind of application legends are made of. Written many years ago, it has been performing well enough to be forgotten about and ignored. The original developer and documentation are long since gone, and it required a great deal of asking around just to find out who used this software and why. Unfortunately, the software, which had been running smoothly for years, did not transition well onto the new servers.

One of our developers (not me) was tasked with getting this service up and running on the new architecture. After spending a few hours digging through the old source code he was able to get it compiling and passing a few basic test cases. We figured that we could move ahead with a production deployment and shift our focus to more pressing matters. We figured wrong.

The next day, the support calls came in once more. Our software worked well enough for a few hours, but something had gone wrong overnight. The application was fully locked up and unresponsive to any requests. A quick restart did the trick, but this was only a temporary band-aid fix and was not acceptable as a long-term solution. The same developer began trying to reproduce the behavior and fix the problem for good. This became the routine over the next few days.

Now, the part I haven't yet told you is that this system has long since been replaced with a newer, shinier, and more fully-featured solution. This new solution is still active with many customers and regular maintenance and enhancements. Only a handful of customers remained on the old system. The reason for staying was simple: don't fix what ain't broke. This system continued to serve their needs for all these years with no updates or known issues.

The way we saw things, we were left with three choices:

• Install the old server hardware back into our production systems to support these customers
• Assign developers to debug the old code and get it working on the new hardware
• Offer these customers the choice between upgrading to the new system or lose this service (with a reasonable amount of time to decide and transition their systems)

The idea of reverting back to the old hardware was dismissed pretty quickly. The reasons we decided to upgrade our hardware were related to security, consistency, and future maintenance. We did not want to have to support 31 flavors of hardware in our production environment.

The remaining choice seemed to come down to simple math. Weigh the revenues against the costs and you'll find the correct decision. Is the math really so simple? How do you come up with a formula for this. I don't know the answer myself, but I'll take a shot at offering a few observations.

Let's start with some obvious factors. Software developers don't come cheap, and having developers spend a few days fighting with a bug can quickly run into the hundreds and even thousands of dollars. Estimate the required effort in time and multiply that by the developer's pay rate and come up with the cost of fixing this bug. Losing a paying customer is contrary to the goals of running a successful business. Continuing to support this solution means that the company can continue to feed off of these revenue streams. Keep in mind that these customers have been using these services with minimal maintenance for many years. Multiply the regular fees by the expected service time to calculate revenue. As with all things, estimates may not accurately reflect reality.

Simple revenues and resource costs don't really cover some of the more complicated factors. If cancelling the service means losing a customer, it may also mean losing future streams of revenue from that customer. When the customer needs to expand their services in the future, who will they turn to? Will they look to maintain a relationship with a company that they already know and trust? If so, terminating that relationship today may mean waving goodbye to new opportunities in the future. What about word of mouth? Damaging your company's reputation may have wide ranging effects in one of the most powerful and underestimated forms of marketing. What about future bug fixes and maintenance? Who is to say that the next round of updates won't expose more bugs in what becomes a money pit of maintenance work for only a handful of customers? What about the confusion caused by the need to support two similar but disparate systems?

To be honest, I am no expert on this matter. The choice was not mine to make and we decided to pull the plug on this legacy application. From a development perspective, I was pleased with this choice. Out with the old and in with the new, I say! I'd much rather have everyone up to date and on the same platform instead of trying to support a creaky old piece of legacy software. From the perspective of a businessman, I'm still not sure.

Have you or your company pulled the plug on legacy software? What were the circumstances? What triggered your decision? What factors did you consider before making the decision? Let me know in the comments.

Cheers,

Joshua Ganes

Backwards Compatibility III - Planning For The Future

Dreaming Big

I like to daydream about my software projects from time to time. In my dreams, my software would become a worldwide sensation. It would be run on millions of computers operated by millions of users using various platforms and each having unique needs. My software would interact with thousands of third-party applications. Of course, my software would be flawless. Nonetheless, I would be forced to pander to the inept masses whose software is not worthy to interface with my magnificent technical achievements.

Down to Earth

In reality, most software that is written today will not see such widespread adoption. There are many software developers earning a reasonable living writing in-house software applications for use by a handful of employees in their own office. Others may have a wider impact, but will not go beyond a few thousand users in their niche market. While backwards compatibility is important for all developers to understand, it is rarely the highest priority for these types of software projects.

One example of software targeting the kind of compatibility required on a global scale is web servers. Web servers must be able to interact with an incredibly wide variety of devices and applications. They should also run on as many varied systems as possible to avoid limiting themselves to a single market. A web server needs to provide your grandmother's garage-sale Pentium II running Internet Explorer on Windows 95 with pictures of her family. It needs to provide content to yuppies browsing social sites on their iPhones during Saturday brunch. It needs to allow 14-year old Linux fanboys to post to their favorite hacker forums.

Identify Yourself

Perhaps the most straightforward way to plan for future compatibility is to add an identification protocol to each interaction. This lets applications on side of the interface know who they're dealing with and to compensate for known issues in their counterpart.

This is really not so dissimilar from interactions with people. You know not to mention your recent SUV purchase to your tree-hugging hippie friend named Rainbow. You probably avoid talking about your contempt for Justin Bieber to your teen-aged niece. In the same way, you can build up a list of known compatibility problems over time and compensate for them by adjusting your own software to avoid the problem or fix it in another way.

Identifying a unique name for each piece of software is not as simple as it may first seem. There are several key pieces of information required to uniquely identify each application: name, version, platform, and configuration.

The name distinguishes similar products with obviously different implementations and opportunity for discrepancies in behavior. For example, consider web browsers provided by different companies such as IE, Firefox, Chrome, etc.

The version is generally a sequential numbering system that can help distinguish older (buggy and limited) versions from newer (corrected and fully-featured) versions.

The platform describes the operating system and architecture on which a cross-platform product is running. This distinguishes the same product built for 32-bit Windows XP, 64-bit Windows 7, or 64-bit Ubuntu Linux.

Finally, the configuration may contain additional information about modifications, plug-ins, and settings that enable special features or trigger certain buggy behavior. This information can be very wide ranging, complex, and difficult to fully capture.

Once each application has verified the identity of its neighbor, it is able to accommodate known compatibility issues and limitations by changing its own behavior. Unfortunately, not all players are always working towards a common goal. These version identification techniques can be turned on their heads and used to limit or disable certain applications.

Expect New Features

With most software, there is a great incentive to add new bells and whistles and little incentive to trim the fat. New features are used to attract new customers and provide new capabilities. Old features may be missed if removed, and are generally simpler to retain than to completely remove.

A widely-used standard designed with this in mind is XML. This, of course, stands for Xtreme Markup Language (or eXtensible Markup Language if you're a real stickler).

XML is designed as a tree of nested tags with attributes and text. Well-designed XML libraries provide the ability to parse XML text into a data structure and to query for desired components within the tree. The beauty of this design is that adding new tags should not interfere with old logic. Older applications are completely free to ignore tags they do not understand and simply proceed as if nothing is different.

By using XML or similar extensible formats for data interchange, applications will be better prepared to accommodate any of the inevitable new features the future has in store.

Be Prepared

As the Boy Scout motto says, "be prepared." If you are designing a software product with the dreams of being compatible with a diverse range of other products and amenable to new features, you must plan for the future. A little planning up front will save you a great deal of grief later. Expect that incompatibilities will crop up and need to be addressed. Expect that new features beyond the original design will be required. Use an interface that is designed to help deal with these problems even before they arise.

Keep in mind that designing for the future can be challenging and time consuming. If you are writing an internal application with no plans for data exchange and targeting a single platform, you're probably best to stick with the "You Aren't Gonna Need It" (YAGNI) approach. Make sure you understand which type of application you are creating before you dive in too quickly and shoot yourself in the foot.

Cheers,

Joshua Ganes

Bowling Is Not A Sport

Rules To Live By

Back in 2003-2004  I completed an internship with IHS Energy in Calgary, Alberta. Not to spark too much controversy, but as an Edmontonian at heart, I cannot advise anyone else to do the same. The company had a lot going for it, many of the people were great, but I couldn't get over the fact that I felt like I was a double agent living within the walls of the enemy base. For those who have not yet sworn allegiance to the Edmonton side of the battle of Alberta, it was a great experience.

While I was there, I was privileged enough to work with a bright and charismatic coworker named Demetrio (I hope I've spelled that correctly). Originally from the Bronx in New York, he was one of those people that everyone seems to like instantly. I admire people with that gift and wonder where that special quality comes from. I came to see him as somewhat of a mentor over my time in Calgary.

Among work-related talks and intense games of foosball, Demetrio would share nuggets of wisdom and humor. On one such occasion, he shared that he and a few of his college friends each came up with a list of three rules to live by. I can no longer remember his first two rules, but the third was so profound that it has stuck with me to this day: "bowling is not a sport".

Obviously, this rule was meant in jest. Yet, somehow, I couldn't get the idea out of my head. Meditating on this thought led me to ask the question, "How do I define a sport?" This question is tougher than it seems. Pause here for a moment and think about your answer it before proceeding.

(I'll wait here ...patiently)

Stay tuned and I will share my thoughts on this important topic.

A Spectrum Of Games

There are some games that the vast majority of people would define as a sport. Whether or not you enjoy it, very few would claim that football (European or American, take your pick) is not a sport. Balanced competitions between opposing teams with a great deal of strength, speed, agility, and strategy are almost universally accepted as sports.

At the other end of the spectrum are games that have no physical component to them, but may contain some amount of strategy. Most board games and card games fall within this category. Indeed, some of these games require great time and commitment to master. Chess is the definitive example of a complex challenge for great minds. Some people include these games in their definition of sport, but most will protest their lack of a physical component.

In the middle lay games and competitions whose validity as a sport are hotly contested. Is, as Demetrio would deny, bowling a sport? What about equestrian events? The jockey clearly has skill and athleticism, but the heavy lifting is done by the horse. What about car racing? Again, drivers have great skill and a long race can be a great test of endurance and concentration. What about individual contests like golf, where the player faces the course, not any opponent. What about judged competitions such as figure skating, dance, or gymnastics?

Formal Definitions

The definition of the word 'sport' as found on dictionary.reference.com contains a list of activities that I would not personally include using my definition of sport (more on that later). It even includes Demetrio's maligned bowling in its list of sports.

The International Olympic Committee recognizes sports as activities that are based in physical athleticism or physical dexterity. This guiding rule seems to be widely accepted by many.

My Definition

To find my own definition of sport, I examined games and competitions that I thought were not deserving of the title. My first target was games without a physical component. If the game can be adequately simulated by a computer, that is a bad sign. If an able-bodied man in his twenties has no advantage over a seventy-year-old retiree with arthritis, I cannot consider the game to be a sport. This rules out most board games and card games. Sorry poker, I love you, but the sports networks are just plain wrong.

I generally like the definition from the International Olympic Committee, but I feel it is still too broad. Prepare yourself for my first controversial criterion. True sports must include an element of strategy. Part of the joy of being a sports pundit is speculating on how I would do things differently if I were calling the shots. Nobody ever told a short-distance sprinter to hold back and lull his opponents into a false sense of security. No, the only strategy is to run as hard and fast as possible.

Another requirement is that the structure of the game must seek to put opponents on an even footing. By this, I mean that a game is designed to be symmetrical with both competitors trying to achieve equivalent goals. While games that break this structure are rare, this requirement becomes necessary for some of the more esoteric competitions out there. This eliminates most events from American Gladiators.

My final rule, which may also be controversial, requires a form of direct opposition. While one of the competitor tries to accomplish a physical task, the opponent must have a way to try to stop or impede his progress. A great deal of the glory of sport derives from a struggle of man vs. man. This rule puts the final nail in the coffin of bowling's claims to sporthood in my mind.

Applying The Tests

My criteria for a sport are quite clear and easy to apply. Even so, there are some events which sit on the border line. The first one that comes to my mind is curling. Let's take a closer look:

1. Curling requires weight control, precise aim, and physical sweeping. CHECK
2. Curling has many strategic components from guards, to freezes, to raises. CHECK
3. Each team has the same number of players and stones and uses the same playing surface and target. CHECK
4. The only "direct" opposition consists of sweeping stones out the back half of the house. check?

Is curling a sport? I'm still not sure.

Please Don't Hurt Me

Before I find myself impaled by javelins, being thrown across the room by weightlifters, or having my face used as a surface for ice dancers, I would like to express my respect for athletes of all forms. You train long and hard to achieve excellence and distinguish yourself in your field. You perform fantastic feats of of strength, skill, and elegance that are exciting and inspirational. Please take my writing with a grain of salt.

My definition of sport comes from the concepts that comes to my mind when I hear the word. I picture hockey, baseball, football, soccer, tennis, and wrestling. These other competitions are wonderful too, but they just seem different to me. For those who see things differently, more power to you.

What Do You Think?

I asked you to come up with your own definition of a sport first. Now that you've heard my theories, how well does it hold up? What do you disagree with? Is there something I've missed?

Demetrio, if you're reading this, I hope you've kept the faith. Thank you for all the good advice.

Cheers,

Joshua Ganes

Stop Whining And Get Over Yourself

A Disturbing Theme

There has been a recurring theme in programming blogs over the past decade or so that continues to get my ire up each time I encounter it. Some entries tiptoe around this theme without ever coming out and state it. Others state it so boldly that I am left with no choice but to roll my eyes or to fume at them and leave a nasty YouTube-style comment. Many authors with incredibly wide readership and influence have repeated this theme using different words.

Can you guess what the theme is?

Maybe a few quotes will help:

• "I think all software is crap" - Steve Yegge
• "all my software is shitty" - Jeff Atwood
• "Software is Crap" - davmac
• "it's all a mess. Everyone sucks, equally and completely." - Scott Hanselman

Shall I go on?

It seems that everyone with an opinion and access to the internet is writing a post bemoaning the sad state of affairs in the world of software. They complain that every single system is utterly flawed at the deepest level and is at risk of crashing immediately if we dare even look at it the wrong way.

A Dose Of Reality

Now, don't get me wrong. There is software out there that is crap. I've even written some of it myself. I feel, however, that saying that all software is crap, or even that most software is crap is far from the truth.

In the process of creating this post I have already interacted (both directly and indirectly) with many different software systems. I used my operating system, its user interface, and various hardware drivers to launch and control a web browser. I used my browser to search for and visit many different websites. Each of these sites is running at least a web server, and most have additional scripting and database systems that work with it. All of that data passes through multiple systems in transit over the internet. Guess what? I have not encountered a single noticeable error up to this poin...NO CARRIER

...

To my readers old enough to get the above reference, I apologize for subjecting you to such a terrible joke. To those younger readers who are scratching their puzzled heads, I apologize for subjecting you to such a terrible joke that you did not even understand.

The fact of the matter is that most of us use a wide variety of software each and every day. Occasionally we encounter a problem that interrupts us or prevents us from accomplishing the task at hand. It may be the blue screen of death disrupting our career-defining presentation to a high-profile client, or our browser freezing halfway through a video of adorable cats diving head first into slightly-too-small boxes. The rest of the time, our software is quietly plodding along in the background and doing its job so well that we barely recognize it's doing anything at all.

It's like that old caretaker at your company that most people were aware of, but nobody could quite tell you what he did. He worked after hours when everyone else had gone home for the night. You might have run into him that time you returned to collect something you had left in your office. Maybe he vacuumed the hallway outside your door that time you stayed late to finish your essential project before the deadline. When he retired, everyone suddenly noticed how so many subtle niceties around the office had changed . When a system is working well, we may not even realize how much we take it for granted until it is gone.

Most software that gets noticed is like the annoying guy at the office who all the managers praise, but nobody wants to work with. He feels the need to add his input on everything and boasts about his "expertise". Trying to integrate his work with the rest of the team sets everyone back as they struggle to correct all of the problems he's caused. He manages to assign the blame to the soft-spoken fellow who sits by himself in the lunch room. When he manages to take credit for the eventual success of the project and parlay it into a promotion, everyone else on the team breaths a grudging sigh of relief that at least they don't need to deal with that guy anymore. This is the bad kind of getting noticed.

The fact of the matter is that for every annoying software system that stands out in the wrong way, there are dozens of others that you barely even notice. You don't notice them because they perform their designated task in silence while you're busy watching groan-inducing videos of skateboarders landing crotch first on the railing instead of stylishly grinding down it.

Professional Pride

I don't know about everyone else, but when I step back from a challenging project that required lots of hard work, time, and energy, I take a certain amount of pride in it. I enjoy seeing efficiency improve as people use my system to accomplish tasks that used to take an order of magnitude longer and require tedious, manual effort. I enjoy creating new software realities out of mere ideas and possibilities.

In any system large enough to be proud of, there will inevitably be some problems. When the bug reports roll in, I don't just throw in the towel and bemoan the fact that I've created yet another crappy system. No, I roll up my sleeves and get to work. I fire up my IDE and get started debugging my code and developing a fix. Depending on the operating environment and the nature of the bug, I can have a fix ready and deployed within hours - delivering a working solution to an eager customer who is (hopefully) pleased with such prompt resolution for their issue.

When I watch users struggle to control my software, I don't just sit back and whine about how I've created yet another clunky user interface. I go back to the drawing board and come up with new ideas on how to make the user interface easier and more intuitive. I perform usability tests with ordinary folks who are trying my software for the first time. I iterate over new designs until I find a solution that is user friendly and simple to understand.

What They Really Mean

Perhaps I'm taking these people too literally. If their point is to acknowledge that most software has bugs and it's nearly impossible to create "perfect" software, then they've got it right. Developing software is challenging. Just when you think you've achieved a new level of competence, a new bug rears its ugly head. The bug mocks you as you sweat and furrow your brow, struggling to figure out what has gone wrong. When you finally figure it out, panting and exasperated, you smack your forehead for overlooking such an obvious flaw.

The good news is that with each boneheaded bug you create and fix, there's actually a chance you might be able to avoid making the same mistake again in the future. If you keep your head down and work hard for many years, you may just learn to create software that isn't a flaming ball of garbage. That said, even the best of us have days when our brains are just not on their game and we write a fancy version of while( 1 ) fork(); Don't lose faith. Keep going and continue striving to become a better developer. In time, you will gain a new level of competence and, at very least, you'll know enough not to let your guard down the next time you're feeling like a project was too easy.

Cheers,

Joshua Ganes

Is It Easier To Write Code Than To Read It?

The Premise

A recent Programmers Stack Exchange post reminded me of a seemingly widespread belief among programmers - writing code is easier than reading code. In his post, "Things You Should Never Do, Part I", about not writing software from scratch, Joel Spolsky writes, "It’s harder to read code than to write it". Most other posts I have found on this topic seem to reference Joel's article. They nod along in agreement as if to say that, once revealed, this premise is gospel truth and cannot be denied.

Horror Stories

Any programmer who's been around the block a few times can remember some godawful snippet of source code that warped their minds and nearly broke their will. Foreign (AKA not mine) source code may be difficult to understand for a variety of reasons. It may be oddly formatted, use poorly named variables and functions, have confusing organization, or seriously lack comments. Other than code full of "clever" tricks (why multiply by 8 when you can just use the bit shift operator?), I believe the most difficult code to read is simply at a level of complexity beyond the expectations of the reader.

Since everyone seems to recall a story about an undecipherable code base from their past, it becomes easy to latch onto Joel's premise. We know that if we aren't careful about our code choices, it can grow into an unholy mess full of badly-named functions full of side effects that run over multiple screens. It's easy to get code into this state and it's difficult to deal with maintenance once the damage has been inflicted. In this sense, Joel's premise is correct.

Defining Difficulty

If we ignore "bad" code and focus on "good" code, I believe that the picture changes. Writing good code is hard. Choosing meaningful and descriptive names for variables and functions is difficult. Structuring and documenting code so it's easy to understand and modify is a constant struggle. Self-documenting code contains all kinds of clues about the purpose of each component and demonstrates how each piece interacts with the other.

I think that part of the problem comes with the vague way we define difficulty. What makes one task hard and one task easy? There are several ways to measure difficulty. Some are more meaningful than others.

I think some of us choose to define difficulty based on our interest in performing a task. Clearly, watching TV is easier than washing the dishes. This measurement breaks down when a task is both challenging and interesting. I'd much rather climb a rugged mountain trail than take out the garbage, but scaling the mountain provides a much bigger challenge than dragging the trash to the corner. Programmers love writing code because of the intellectual challenge and the joy of creating something from nothing. Understanding the work of another coder does not generally hold the same appeal.

When it comes to business, a much better way to measure the difficulty of a task is in the cost in resources. In this case, we're talking about the time and effort of a programmer. It may not be fantastically interesting to decipher someone else's code, but given a well-written code base, understanding comes with perseverance. Rewriting the same piece of code (larger than a few hundred lines) to the same level of quality will require an order of magnitude in extra time.

Discipline

Here's the bad news: every field has unpleasant tasks. As a programmer, I prefer writing new code over maintaining old code. When someone proposes an unpleasant task, people tend to come up with all kinds of excuses to avoid it. This is as true for programmers as for anyone else. The excuses pile up until we talk our way out of doing the unpleasant task.

Unfortunately, as professional software developers, we're not being paid for pleasant. We're being paid for results. Great developers recognize this and exercise discipline. They buckle down and read the code carefully. They are able to understand it much more quickly than it could be rewritten. Once they understand the code, they make their changes and move on.

The good news is that if you follow this approach, you'll get things done quickly and be able to move on to new projects. Reading good code instead of rewriting it will result in a higher level of productivity. Your boss may just take notice and reward you for your efforts.

Cheers,

Joshua Ganes