Make tools for yourself and others around you

There’s a joke in computer science that everything can be solved with another layer of abstraction or indirection. It’s true. In our line of work, tools are essentially good abstractions for the problem we’re solving. Our craft and discipline is built with abstractions, and yet, I find it’s something hard for programmers to grasp.

Why use abstractions in the first place? We build abstractions to make it easier to work on the problem we’re trying to solve, without getting distracted by details from the big picture. We want to be able to drive a car using a an abstraction called a gas pedal, rather than timing the firings of the spark plugs in the engine, which is what actually moves the car forward. This abstraction lets us worry about that “what”–getting from point A to point B–without worrying about the “how”–the specifics of how that occurs.

I think we all know the above logically, but emotionally and in practice, we often disregard the ideas above. Ideally, a programmer not only gets the program to do what it wants, but also produces usable abstractions for other coworkers along the way. Often times we laud the quickest programmers, but I think the best programmers are ones that build usable abstractions for other programmers. The ones that build tools for others to use and increases the productivity of those around them. And yet, I’ve met programmers that didn’t like to build abstractions.

There are some programmers that like to have everything on a single page. . I suspect these are programmers that can hold an inordinate number of things in their head. They can look at all the details of which pistons are firing at which time all at once, and still understand the whole thing. But even this is unsustainable at some point, as there’s no physics capping the upper limit on the size of code bases. In addition, coding, at its core, is a social activity. If you write code that others find hard to follow because you’re mixing the “what” with the “how”, then your code is less useful to others, both to use and to maintain.

There are yet other programmers that fear it makes the code slow. I’ve seen Ruby code written as if it was C code. This is counter productive. Often times you don’t know which part of your code is the slowest until you measure and profile. Often times, your intuition is wrong about which paths get executed the most. There are diminishing returns to make a piece of code faster. If it takes 10 seconds to run, then a 50% reduction in time is great! But if it already takes 10 milliseconds to run, then the extra 5 milliseconds may not be worth it, as there are often many other pieces of code that would take 5 milliseconds or more to run. In addition, a user will not notice 5 ms as much as 5 seconds.

So while not every programmer should build tools for others, I believe it’s something every good programmer should aspire to. However, this is often a difficult skill to learn and get right in practice. It’s not easy to do under the time crunch of deadlines and fire drills. Even if you didn’t have those time pressures, it’s still hard to get right.

Why is it so hard to build useful abstractions? For one, you can often hide the wrong details. And when you hide the wrong details, the abstraction is useless because you need to manipulate the details. On the other hand you can veer too far the other way, by building abstractions that are too flexible, requiring too many new concepts for users to grasp before they’re productive.

The culprit to both problems is that we often don’t understand the problem well enough when we sit down to write. When you don’t yet understand all the different circumstances that you’d use an abstraction, you expose either too much or too little.

There are two ways to combat this. First, you should set out to write the application, and extract the framework or platform for it, rather than setting out to write the platform. I’ve seen some useless web frameworks because they set out to write the framework, rather than be informed by experience writing applications. Second, you should do something akin to sketching. You write the core of what is still unknown about the problem domain then test it. If you’re building a blog engine, you start writing the posting and commenting, not the login. Then refine it as you understand more about the domain.

Make tools for yourself that’s informed by the problems you face when you’re trying to get something done. It’s not a waste of time. It’ll make you sharper as a programmer, and you may get better technical or business ideas as a result. It’ll make you more productive, and you’ll be able to delight others once you learn this skill that requires experience and taste.


5 thoughts on “Make tools for yourself and others around you

  1. I always phrased this in terms of a “capability-oriented” vs “product-oriented” dichotomy.

    The “capability-oriented” approach seeks to maximise the capabilities of the organization over the long term, opening up opportunities for cost amortization over disparate products. By contrast, the “product-oriented” approach is “lean” in the sense that it seeks to minimize work not absolutely necessary for the current product.

    The “product-oriented” approach seeks a local optimum, and is prevalent in industries with heavyweight cost controls and minimal opportunities for investment (Defense, for example), whereas the “capability-oriented” approach seeks a global optimum, and is only possible in situations where organizational leadership is capable and willing to get involved with detailed technical planning.

    As with all dichotomies, organizations must seek a compromise position that varies from product to product and project to project.

    A sweet-spot that I have frequently found: Third-party libraries are necessarily written in a very general manner, capable of being used in a wide variety of ways. A particular organization generally only uses a small subset of the features in a particular third-party library. As a result, an internal library that acts as a thin wrapper around an external/third-party library can help encourage and enforce a consistent approach within an organization, helping to create an institutional culture and improving interoperability between teams whilst reducing learning barriers for new team members.

    1. 3rd party libraries are good to use, provided that you do your homework. But on the whole, I also see dichotomy you laid out. There are definitely times where you’re sketching and trying things out, where you shouldn’t worry about doing everything the right way. I originally did try to address both looking at stack down the abstraction stack as well as up, and also when to make tools, and when to just get it done. But that kind of article is unwieldy, so I decided to just focus on one aspect.

      Ideally, you should be able to shift between the two easily, but I don’t think many languages think about how to move from sketch to inked. The only thing I’ve seen are optional static typing in Go, iirc.

      1. I agree that there exists a trade off.

        You can either tackle the problem at hand by the most direct route possible, (your “sketching”, my “product-oriented”), or you can try to build tools & abstractions to help future development efforts be faster and more efficient (your “inking”, my “capability-oriented”). The route/weighting that you choose is driven both by immediate constraints and long term goals, and how you balance one against the other.

        This is sort of related to predictive/CMMI vs reactive/agile in that the reactive/agile approach is well served by a capability-oriented toolset.

        There is no “right” answer, it all depends upon the situation that you are in and the culture of the organization(s) with which you are operating. One thing that is really important however, is motivation. Software is all about detail, and detail is all about attention, and attention is all about motivation, and motivation is all about the team. Maintaining motivation and enthusiasm in the team just about trumps everything else.

        You should do what you enjoy, because you will be so much more productive that any other consideration will be secondary.

    1. Nope. The 3D printer post will come later. There’s both a lot of hype and cynicism around it, and I figured I’d write something that told people what was going on at the ground level.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s