Make tools for yourself and others around you

There’s a joke in computer science that everything can be solved with another layer of abstraction or indirection. It’s true. In our line of work, tools are essentially good abstractions for the problem we’re solving. Our craft and discipline is built with abstractions, and yet, I find it’s something hard for programmers to grasp.

Why use abstractions in the first place? We build abstractions to make it easier to work on the problem we’re trying to solve, without getting distracted by details from the big picture. We want to be able to drive a car using a an abstraction called a gas pedal, rather than timing the firings of the spark plugs in the engine, which is what actually moves the car forward. This abstraction lets us worry about that “what”–getting from point A to point B–without worrying about the “how”–the specifics of how that occurs.

I think we all know the above logically, but emotionally and in practice, we often disregard the ideas above. Ideally, a programmer not only gets the program to do what it wants, but also produces usable abstractions for other coworkers along the way. Often times we laud the quickest programmers, but I think the best programmers are ones that build usable abstractions for other programmers. The ones that build tools for others to use and increases the productivity of those around them. And yet, I’ve met programmers that didn’t like to build abstractions.

There are some programmers that like to have everything on a single page. . I suspect these are programmers that can hold an inordinate number of things in their head. They can look at all the details of which pistons are firing at which time all at once, and still understand the whole thing. But even this is unsustainable at some point, as there’s no physics capping the upper limit on the size of code bases. In addition, coding, at its core, is a social activity. If you write code that others find hard to follow because you’re mixing the “what” with the “how”, then your code is less useful to others, both to use and to maintain.

There are yet other programmers that fear it makes the code slow. I’ve seen Ruby code written as if it was C code. This is counter productive. Often times you don’t know which part of your code is the slowest until you measure and profile. Often times, your intuition is wrong about which paths get executed the most. There are diminishing returns to make a piece of code faster. If it takes 10 seconds to run, then a 50% reduction in time is great! But if it already takes 10 milliseconds to run, then the extra 5 milliseconds may not be worth it, as there are often many other pieces of code that would take 5 milliseconds or more to run. In addition, a user will not notice 5 ms as much as 5 seconds.

So while not every programmer should build tools for others, I believe it’s something every good programmer should aspire to. However, this is often a difficult skill to learn and get right in practice. It’s not easy to do under the time crunch of deadlines and fire drills. Even if you didn’t have those time pressures, it’s still hard to get right.

Why is it so hard to build useful abstractions? For one, you can often hide the wrong details. And when you hide the wrong details, the abstraction is useless because you need to manipulate the details. On the other hand you can veer too far the other way, by building abstractions that are too flexible, requiring too many new concepts for users to grasp before they’re productive.

The culprit to both problems is that we often don’t understand the problem well enough when we sit down to write. When you don’t yet understand all the different circumstances that you’d use an abstraction, you expose either too much or too little.

There are two ways to combat this. First, you should set out to write the application, and extract the framework or platform for it, rather than setting out to write the platform. I’ve seen some useless web frameworks because they set out to write the framework, rather than be informed by experience writing applications. Second, you should do something akin to sketching. You write the core of what is still unknown about the problem domain then test it. If you’re building a blog engine, you start writing the posting and commenting, not the login. Then refine it as you understand more about the domain.

Make tools for yourself that’s informed by the problems you face when you’re trying to get something done. It’s not a waste of time. It’ll make you sharper as a programmer, and you may get better technical or business ideas as a result. It’ll make you more productive, and you’ll be able to delight others once you learn this skill that requires experience and taste.

Advertisements

How to stream output of system process in Ruby rake task?

Often time, I use Rake to manage a number of tasks I use often, but I can’t remember the command. It’s part of building tools for yourself to make yourself more productive.

Sometimes, you use rake tasks to run servers, like so:

desc "Start the server"
task :start do
  puts `node app.js`
end

However, that doesn’t work. The child process blocks, and won’t return until the server exits, so that means `puts` never gets a return, and hence never executes. In addition, stdout of the child process running node isn’t connected to the stdout of the parent process running rake. Therefore, you won’t see any output from the server.

Finally, being fed up with the situation, I started looking around on google and stackoverflow, and it’s kind of an elusive question. But I finally have it. It looks like this:

desc "Start the server"
task :start do
  IO.popen("node app.js") { |f| f.each { puts line } }
end

As long as `node app.js` continuously flushes its stdout, we’ll get its output sent to the rake’s stdout. Yay. One less annoying thing to deal with.

[1] Reference: http://stackoverflow.com/questions/1154846/continuously-read-from-stdout-of-external-process-in-ruby

Dalton’s point

Recently, Dalton Caldwell posted an Open Letter to Mark Zuckerburg. I was surprised and disappointed by this top level comment and others on Hacker News:

I don’t understand what you think Facebook did wrong. They intend to enter this new space, using their technology and their people. As a courtesy they offered to hire you / throw a lot of money at you. What would you have had them do instead? Not compete with you, because you’re a precious snow flake? Acquire you and treat you like a prima donna, giving you your own team and allowing you to take your own technical direction? – tjlc

I don’t get it either. Last week this guy wanted to make some sort of paid Open Twitter competitor. This week he’s mad at Facebook for offering to acquire his company rather than simply build a first party feature! – temphn

Then the rest of the thread were people arguing over the more granular points about hard-ball tactics. Or about whether it was kosher to invite someone to a meeting and then change the topic of discussion. Or whether a company has a right to do whatever they please with their technology.

It’s as if many readers read one part of the article and it offended some resemblance of ‘sensibility’ in them that they had to click the back button and start ranting about things that have nothing to do with the main point, and others take the bait to feed the trolls.

His open letter to Mark is not about being acu-hired at all, nor is it at all a public complaint what Facebook did, or any of the topics mentioned above. For a while now, the main thesis of his recent blog posts is what he calls an audacious idea:

Social media platforms should not be ad-supported.

Or, if you like it in his own words:

“My thesis is that if you purport to be a platform, your business model needs to reflect your platform-ness. Because if you instead have an ad business model, and you’re not making enough money, you can end up slaughtering your ecosystem.” – Dalton Caldwell

This blog post is a continuation of that theme. The TD; DR version of the open letter is:

Facebook employees are doing asshole things to 3rd party developers[1], even though Facebook calls itself a social media platform[2], not because they’re bad people, but due to bad incentives[3] caused by a search for ad-revenue[4].

[1] “Mark, I know for a fact that my experience was not an isolated incident. Several other startup founders & Facebook employees have told me that what I experienced was part of a systematic M&A “formula”. Your team doesn’t seem to understand that being “good negotiators” vs implying that you will destroy someone’s business built on your “open platform” are not the same thing.”

[2] “Mark, based on everything I know about you, I think you get all of this. It’s why you launched FB platform to begin with. Do remember how you used to always refer to Facebook as a “social utility”?

[3] “I don’t think you or your employees are bad people. I just think you constructed a business that has financial motivations that are not in-line with users & developers.” – Dalton

[4] “your “platform developer relations” executive made no attempt to defend my position. Rather, he explained that he was recently given ownership of App Center, and that because of new ad units they were building, he was now responsible for over $1B/year in ad revenue. The execs in the room made clear that the success of my product would be an impediment to your ad revenue financial goals,” – Dalton

In fact, I’d venture to say that he’s not so much mad at Facebook, or even really caring that Mark Zuckerberg reads it or not. It’s like if you had concerns about the direction education in America, you’d write an op ed piece that is creatively written as a letter that starts with “Dear President Obama”. Of course, you don’t actually expect President Obama to read it, but you do expect Americans that read it to understand your concern for the status of education in America and address your main thesis.

For Dalton, I suspect it’s another piece of proof for him that he’s on the right track about how social media platforms shouldn’t be ad-supported. Not only that he’s looking for other people that believes this thesis.

“The only way to get fulfillment is to help other people that believe what you believe.” – Simon Sinek

And I should add, Simon is very specific about what he means by “believe”:

“We want to go to work with people who understand us, who believe what we believe, who have a similar view of the world, that has nothing to do with their opinions and the differences that we share. That’s good. that’s called diversity. That’s called advantages to problem solving, which is we can all look at the same thing from different angle and come up with solutions. What I’m talking about is why should you help each other in the first place. What are you in the pursuit of?” – Simon Sinek [emphasis mine]

Dalton is in the pursuit of this crazy (gasp!) idea about a non-ad supported social media platform, and blogged about it repeatedly [a] [b] [c] [d]. He’s trying to find people who believes what he believes. He is proposing something that goes against the current tide and grain, because he believes it’s the way forward. He believes it’s something sustainable, a public good, and the springboard for further innovation. And when you find yourself in that situation, you need to find other people that believes what you believe to help you realize the vision of the future that betters us all.

Google services being axed? Sounds like a list of startup ideas.

Google services are being axed left and right, due to Larry Page’s continual focus on what-matters to his company, via advice from Steve Jobs. It’s not bad advice at all, and I think companies have cycles of both experimentation and focus. It just happens to be in the focus phase.

And when Google decides to axe a product, it’s probably worth asking why they axed it. Is it because it’s untenable or it’s no longer needed, like iGoogle? Or is it because it no longer fits into the strategy of company?

Google Bookmarks Lists, Google Friend Connect, Google Gears, Google Search Timeline, Google Wave, Knol, Renewable Energy Cheaper than Coal (RE<C), Aardvark, Desktop, Fast Flip, Google Maps API for Flash, Google Pack, Google Web Security, Image Labeler, Notebook, Sidewiki, Subscribed Links,Google Flu Vaccine Finder, Google Related, Google Sync for BlackBerry, mobile web app for Google Talk, One Pass, Patent Search, Picasa for Linux, Picasa Web Albums Uploader for Mac and Picasa Web Albums Plugin for iPhoto, and all Slide products.

http://techcrunch.com/2012/07/03/google-shutdowns-continue-igoogle-google-video-google-mini-others-are-killed/

I suspect that some of them here are perfectly good startup ideas that are worth reviewing. What are small potatoes for Google may just be a mountain for you.

What would be the best way for an aspiring programmer with not much prior experience to learn Javascript?

Someone asked me this question on Askolo

The best way to learn programming is to do it. I think there’s two tracks that you need to pursue: learn the fundamentals of programming (aka, the boring, but important stuff), and a project you want to do.


No one likes doing drills, but if you haven’t done programming before, you need the basics, like how to do if/then statements and loops. It’s like having to know what a steering wheel is before you drive. I suggest getting a javascript book for beginners. 
http://www.amazon.com/s/ref=nb_sb_noss_1?url=search-alias%3Daps&field-keywords=javascript+beginners&x=0&y=0

Or if you’re a cheap bastard, you can start with http://www.w3schools.com/js/ which gives you examples you can play with right in the browser! It’s important that you play with the examples. That’s how you get to know how it works! Change something, see what it does!

The other half is to pick a simple project that you’d like to do. The idea is to pick something that’s not too hard for you (important!), but something you’re really really excited about building (double important!). It can be stuff like make every image in a webpage be upside down as a prank for your friends. Or implement this XKCD comic: http://xkcd.com/1031/ I can’t really tell you what to pick, since you need to pick something exciting to you, so you’ll want to work on it, even when your friends ask you out for a kegger. But also make sure it’s doable. That’s hard to judge when you’re just starting out, but one way is to ask a programmer friend that can appreciate beginners. 

After you get the basics from w3schools, I’d say it’s important to learn JQuery. http://jquery.com/It’s the sane was of manipulating the stuff you’d see on the page. 

After that, you’re ready to tackle:
http://ejohn.org/apps/learn/
I think it’s a really good tutorial on the little gotchas of Javascript. It may take a little while to get through it, but feel free to come back to it, as you work on other stuff.

Javascript is a surprisingly good language, but it has lots of warts. Learn about what the good parts are:
http://www.amazon.com/JavaScript-Good-Parts-Douglas-Crockford/dp/0596517742

Remember to do the following when coding:
– Write a little, test a little. That way, you know what you changed since last time that broke it. Don’t try to write the entire thing all at once, and then run it. 
– Make sure you check that you’re not missing semicolons, and all your braces, parens, and brackets match. Computers are finicky.
– If you have questions or get stuck, search on stackoverflow.com They have lots of answers. 


Good luck!

Writing on the web has gone from “listen to me” to “ask me anything”

Writing on the web has taken many forms over the years. The earliest being geocities homepages, but first materialized as a format with blogging. Since then, blogging has spawned a lot of different forms, but adjusting its different properties. Limit the size of the message and make following easy, and you get Twitter. Make posting multimedia and reposting it easy, and you get Tumblr. Make it only about posting pictures on the go with filters with easy sharing on Twitter and Facebook, and you get Instagram.[1]

The other way to approach the problem of getting people to write on the web is a question and answers site. Before Quora, I honestly never gave much thought to Q&A sites, since all the ones I saw were pretty terrible, like Yahoo Answers. But when Quora came out a little over a year ago, I found it fun to read, since the answers tended to be thoughtful. Since then, Q&A has taken many forms, from AMA (Ask Me Anything) on Reddit to Formspring. [2]

I’ve not been posting here as much in the past week partially because I’ve been writing on a site called Askolo [3], which is another take on Q&A sites. Askolo is like formspring, where you can answer questions people ask you, but the questions and answers are more thoughtful. I’ve found both writing and reading on Askolo to be addicting to the point I needed to block the site when I’m trying to get work done.

Of course, your mileage may vary, but I found it attractive as a reader because I knew the other users on Askolo, but often times, I discovered things about them that didn’t know and wouldn’t have thought to ask. In addition, I can ask a question, and with enough votes of curiosity from other Askolians, they feel compelled to answer. A lot of the time, if someone was an expert or knew quite a bit about a particular area or field, I could ask them about it as well.

On the other side of the equation, I write on Askolo, because they’re what other people are interested in knowing about me or something I can answer. Curiously, I remembered something that Robert Scoble posted at about how he was wrong about Quora as a blogging service. Because Askolo is directed at specific people, I thought it was a good platform for doing lead gen on what blog posts to write. 

One of the unsaid things about blogging is that it’s a bit of work to write. And you have to do it for a long time in order to see conversations and social returns. In addition, you often don’t know what what topics to write about to generate readership. So the only recourse is to write a lot, about a variety of topics, keep track of what’s trending and write about that. With Askolo, people can vote on questions they’re curious about, so it acts as lead gen for blog post topics people would be interested in. 

As we got better at figuring out what the web was good for, we realized the web is really good at keeping us social with each other, despite naysayers in the 90’s. And as a consequence, it’s not total unique visitors that were important, but it was engagement that was quite important in keeping users around (which is why websites are getting more and more addicting).

That engagement happens when we can interact with others in a meaningful way around some type of digital artifact, whether it’s photos, tweet, blog post, or questions. And with blogs, the longstanding method of interaction was really just comments or writing your own blog post as a reply. So when you can ask someone questions on a blog-like platform, it opens up a new channel of meaningful interaction to drive engagement, and everyone gets more fun out of it.

So I think they’re on to something, but I’m not exactly sure where it’ll go. I think there’s a couple of different potential ways to go from lead gen for blogging to a dating site. As writing on the web has evolved from Blogger, to Twitter, to Quora, Askolo is an evolution in ways to write on the web, and it’ll be interesting to see where the space goes.

In the mean time, feel free to ask me questions on Askolo, or vote on questions you’d like to see answered, and I’ll do my best to answer them. Better answers will make it to this blog as a post. Happy asking!

[1] Of course, I’m over-simplifying. These aren’t the only reasons why those sites are successful.

[2] Quora’s not the first, but I’m too lazy to figure out which out of the three were the first. My point is that they’re all in the same category at around the same time.

[3] In full disclosure, I know the founders, I helped them beta test the site. 

 

Introducing Efronsitive–parametric 3D printable non-transitive dice

Two weeks ago, I found out about non-transitive dice from Hacker News. And I thought it would be a good simple project to build a 3D model for in OpenSCAD, after designing King’s Gambit, an open source 3D printable Chess set

It was definitely a lot easier to design than the chess set, and I managed to do it in an afternoon. I probably spent more time writing up the documentation and this blog post than I did actually designing the thing. 

The code repository is on github here. Feel free to fork the design and try to do other things with it, and send pull requests. 

What are non-transitive dice?

Non-transitive dice are a set of dice where no one dice has a better chance of winning head-to-head with another dice in the set. They’re like the rock-paper-scissors of the dice world.

The idea for them have been around since the 1970’s: Gardner, M. “Mathematical Games: The Paradox of the Nontransitive Dice and the Elusive Principle of Indifference.” Sci. Amer. 223 , 110-114, Dec. 1970.

When you look closely at the dice, they might have more than one face with the same number of pips (the little divets), or eve no pips at all. Some might have numbers above six as well. 

There are different sets of possible non-transitive dice, usually named after its inventor. The three I have implemented here are Efron, Grime, and Miwin. If you have others, feel free to fork, contribute, and send a pull request! 

Efron’s Dice

Screen_shot_2012-02-27_at_11

Efron’s dice are a set of four dice where each dice beats their predecessor by a probability of 2/3.

The dice is named and ordered by the color scheme where:

Blue > Magenta > Olive > Red > Blue

which means if you’re reading the wikipedia article, A corresponds to Blue, B to Magenta, C to Olive, and D to Red. The colors of the dice are in alphabetical order, to help you remember. This color scheme was adopted from here

Miwin’s Dice

Screen_shot_2012-02-27_at_11

Miwin’s dice are two sets of three dice each with the probability that it will have a higher number than another is 17/36; a lower number, 15/36.

Since there are only three dice, each dice will beat one and lose to the other on average.

The dice can be distinguished by color, or by the sum of their smallest two numbers. For example, the III dice in the Miwin set has faces 1 and 2 on it, which are the smallest two numbers on the dice. Add up 1 and 2, and you get 3, which is the name of that Miwin dice. Dices III, IV, and V, are in one set, and dices IX, X, and XI, are in the other.

These set of dice also has reverse non-transitivity. Normally, III beats IV. But if you remove the common faces, the dice reverses their intransitivity, meaning now IV beats III. This applies to the other two combinations of IV and V, and V and III.

Grime’s Dice

Grime’s dice is a set of 5 dice, each denoted by a color, where the order of which dice beats which is ordered in two ways:

Alphabetical order of the color's name
Blue > Magenta > Olive > Red > Yellow > Blue

By the length of each color's name
Red > Blue > Olive > Yellow > Magenta > Red

In general, the alphabetical chain is stronger than the word length chain. Overall, the average winning probability for one die is 63%

Like Miwin’s dice, Grime’s dice also has reverse non-transitivity. If you have two dice of each color, now the order for word length now flips so:

Magenta > Yellow > Olive > Blue > Red  >Magenta

With two dice, the chain ordered by word-length is stronger than the alphabetical chain. The average winning probability for two dice is 59%

Grime’s dice has other fascinating properties that I suggest you look at here.

Lessons Learned

I found that I spent most of my time writing in OpenSCAD doing refactoring. I’m not sure what the problem is yet, but it’s a symptom of some underlying problem with the programming language’s interface with the user. 

In addition, I also learned that you can put modules within another to help organize the code and not pollute the namespace, but there’s no way to access those modules outside of the containing module, so you can’t actually use it to namespace modules. I also leanred how to use default arguments in the module calls for friendlier programmer interface.

I also tried organizing the code so that there’s a src directory and a print directory, just like there’s a src/ and bin/ in software. However, it’s often not as simple as having a main.scad. Often times, demonstrating what it looks like assembled is different than arranging multiple pieces for printing. So I succumbed to the use of plate.scad to indicate that this is the file you should compile to STL to print. Sometimes, you have multiple plates, and you have to do all the compilation to STL by hand (unless I find a way to convert *.scad to *.stl in OpenSCAD on the command line), which makes it tedious. I think eventually, the printer driver software will allow you to arrange plates, but in the meantime, we’ll have to provide both a plate.scad and an assembly.scad.

Rounded corners are tedious to do. There are easier ways of programming rounded corners, by using the convex hull() or minkowski() methods, but both are so ridiculously slow at rendering (especially the latter), that it makes it pretty much impossible to use practically. 

Well, other than that, I’m looking for other simple projects to model. Have fun with the parametric dice, and send me a pull request!