Crazy small nuances

I’m not particularly fond of nuances. Ruby is suppose to be easy to use and intuitive to the user. But as it grows, it has some crazy little nuances.

For example, there’s a difference between system and exec. I wouldn’t have known, as I was skimming the docs for something I needed. Were it not for this post by jayfields, I would have had no idea.

And there’s something in Ruby 1.9 called “funcall”, in which it’s definitely not at all intuitive how that’s different from “send”. I hope these are just growing pains, because while Ruby is nice when you’re taking it out for dinner on the first couple of dates, I hope it doesn’t get abusive the more time that you spend with it.

Users know the problem. You know the solution

What’s considered by now to be mantra out there in ‘user-centric web development land’, is “listen to your users“. But in all that hubbub, you’ll also hear, “don’t listen to your users all that much.”

So what’s up with the paradox? Well, like all puzzles, it’s only a puzzle if you think it’s all or nothing. It seems pretty obvious once you think about it, but it bears reiterating:

Listen to your users when it comes to what’s wrong with your stuff. Users are more familiar with problems with your application than you are. They are pained by how much your application sucks, and how it doesn’t help them get on with their lives. So they’ll complain to you, in hopes that you’re listening and you’ll fix it.

But users often times have no idea what solutions are good. They’ll often offer up solutions to their problems with your application when they’re telling you what’s wrong with it. Often times, they won’t even tell you the problem, they’ll just offer solutions.

“You should put tagging in here.”

“Why not put a chat room so we users can talk to each other?”

However, they usually don’t have the overall vision, sufficient scope, and adequate background for improving the product. That’s your job. You have to get beyond what users are saying to figure out exactly what the problem is, and find a solution that fits the overall vision of the product, and perhaps solves other problems all at once also.

A default behavior for failure or nil

I read “The Rails Way“, mainly because Jamis Buck writes there. Three days ago, they had actually posted one thing that I had address just a few days prior–that is, how to prevent users from looking at other users’ data.

I have to admit, I was rather tickled by my solution. It’s nice when you figure things out. But alas, after reading Koz’s post on association proxies, I have to admit, I like his solution better. At least I made it to anti-pattern #3.

So what did he do? He simply used the find in the association, and let things throw an exception otherwise.

def show
@todo_list = current_user.todo_lists.find(params[:id])
rescue ActiveRecord::RecordNotFound => e
flash[:warning] = "Stop playing around with your urls"
redirect_to '/'

It also takes care of the case where you have say, many items that belong to a todo_list. You can load it by using :condition in the association find. The only reason I can think of not to use it is if it happens to be a slow solution. But no use optimizing if you have no measurements and hard numbers.

Koz’s solution let the default behavior of the method take care of things. This is a way of thinking that I need to start using more of.

I was use to nil being a failure state, something that you checked, and if it happened, everything’s gotta stop–like the examples I often saw in C:

status = CallSomeMethod(with, some, parameters);
if status == NULL {
// or throw some exception here if you're using C++ or Java

Often times, these things got cumbersome, because NULL (or even an empty array or hash) was not considered to be a valid input for many functions, and would stop computation by returning error codes and throwing exceptions in C++.

One of the things that I found nice about the code from Ruby gurus was that it was conventional to do (and subsequently, I saw in Perl):

setting = params[:setting] || "default"

Instead of:

setting = params[:setting].nil? ? "default" : params[:setting]


setting ||= "default"

Instead of:

setting = setting.nil? ? "default" : setting

It was because the operator || took nil by default, and had an appropriate behavior for dealing with nil, and that has made all the difference in being able to chain functions together. In addition, having default failure behaviors require you not have to write error checking code all over the place either.

I’m not sure what this is called (if anyone knows, enlighten me), but a completeness and liberal in what you accept for your input and strict in what you output also applies here to methods and functions. I think it makes code more readable, because it is not peppered with common sense error checking code.

Capistrano and Mongrel are easy to use, but deployment is still hard

So I finally got around to deploying with Capistrano. It was like learning how to walk all over again. Before, I had learned to deploy using just lightty behind apache, and that was a pain every time, since there would be steps I’d be forgetting. A good introduction to deploying with capistrano is “It’s time for a grown up server.” I also checked out Mongrel at the same time. It seems pretty neat.

I’m sure that Cappy is easy to use once it’s up and running, but boy, setting it up on a server from scratch has taken all day. This Yariv’s guy has a point when he says there’s so much stuff to install, compared to web development in Erlang–all you need is Yaws and Mnesia. But then again, all this ‘stuff’ is suppose to make our lives easier.

There are plenty of tutorials out there about setting up Capistrano and Mongrel, but deviate from them just a little, you’d better be ready to use your problem solving noggin.

SVN is just another TLA

One of the biggest problems I’ve run into while setting things up is that if you have a password on your svn repository, Cappy’s not going to like it too much. This is an old problem, and the old solution from Jamis exists. However, I noticed that when I used his solution, the options –username and –password was there twice. I’m guessing that you can do the following in your capistrano deploy.rb, and it would work…BUT!:

set :svn_username, ENV['USER'] || "default_username"
set :svn_password, { Capistrano::CLI.password_prompt('SVN Password: ') }
set :repository, "{application}/trunk"

This only works if the machine you’re developing on is the same as the machine that you’re deploying on, per the example in Agile Web Development with Rails (and already the example is outdated). When I did the cold deploy, I got this:

$ cap cold_deploy
* executing task cold_deploy
SVN Password:
...some other stuff clipped...
** [out :: remote_server] Authentication realm:
** [out :: remote_server] Password for 'someapp':
** [out :: remote_server] subversion is asking for a password
** [out :: remote_server] Authentication realm:
** [out :: remote_server] Username:

It seems like Capistrano checks the SVN twice. Once to find the latest version number when it is on the local development machine, and once again to checkout the latest version from the repository on the remote server machine. Since I had no way of controlling what goes on in the script while being executed on the remote machine, the prompts for a username and password for svn just got logged, and Cappy just waits on the local machine.

The only solution I came up with was just to go back to the Old Ways, where you ssh into the remote server machine and do a temp checkout by hand first. And you do the same on your local development machine.

$ svn co
$ rm -rf trunk

You can then just leave the deploy.rb script as:

set :repository, "{application}/trunk"

This way, svn caches your authentication on the remote machine (and local machine), and probably won’t ask for it again until maybe you reboot.

To get the pack of Mongrels running, you’d better get things right

Things were going hunky-dory, and I thought I was in the clear, but then there was a slight detour. I ran into this problem when I tried to start mongrel on the remote server.

$ mongrel_rails start -d
/usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/rails.rb:32: uninitialized constant Mongrel::HttpHandler (NameError)
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails:10
from /usr/bin/mongrel_rails:18

This one took a little bit of time to figure out, but it was easier than the next two. Gems said that mongrel was successful at the end, when in fact, it didn’t build at all! This was because I didn’t have make on my machine. To get it on Ubuntu, apt-get the build-essential package, and then reinstall the mongrel and mongrel_cluster gems. Odd thing is, I needed make to build rubygems on Ubuntu. Must have got removed when I did apt-get autoremove

However, the real kicker stumped me for a while here:

$ cap cold_deploy
** transaction: commit
* executing task spinner
* executing task start_mongrel_cluster
* executing "sudo mongrel_rails cluster::start -C /var/www/apps/my_app/current/config/mongrel_cluster.yml"
servers: ["remote_server"]
[remote_server] executing command
command finished
command "sudo mongrel_rails cluster::start -C /var/www/apps/my_app/current/config/mongrel_cluster.yml" failed on remote_server

Huh? What’s going on? I was stumped for most of the day. It’s always the little thing that get you, but you learn, and alas you learn. Ubuntu, by default, does not add any subsequent users to sudo. I had created another user on the remote server machine to do the deployment. Therefore, it had no sudo privileges. Run:

sudo usermod -G admin username

And after that, I ran into:

$ cap cold_deploy
servers: ["remote_server"]
[remote_server] executing command
** [out :: remote_server] Starting 2 Mongrel servers...
** [out :: remote_server] !!! Path to log file not valid: log/mongrel.log
** [out :: remote_server] mongrel::start reported an error. Use mongrel_rails mongrel::start -h to get help.
** [out :: remote_server] mongrel_rails start -d -e production -p 8000 -a -P log/ -c /var/www/apps/my_app
** [out :: remote_server] !!! Path to log file not valid: log/mongrel.log
** [out :: remote_server] mongrel::start reported an error. Use mongrel_rails mongrel::start -h to get help.
** [out :: aeolus] mongrel_rails start -d -e production -p 8001 -a -P log/ -c /var/www/apps/my_app
command finished

After reading through some google posts, I realized it was really simple. The configuration file for mongrel had the wrong path to it. It didn’t have the right path. in the mongrel setup, you need to make sure the deploy path as “current” on the end. Remember to add /current/ to wherever you’re deploying your application!

mongrel_rails cluster::configure -e production -p 8000 -a -N 2 -c /deploy/path/my_app/current

So hopefully, if any of you out there has run into the same problems, I’ve saved you a little bit of time. I can’t really say this is a tip, more like a log of my adventures deploying. Remember, if just doing it once will save you lots of time in the future, it’s probably worth it to learn how to do it.

Does link_to_remote() submit using a GET or a POST?

A little while back, I had a post about how to have two submit buttons with form_remote_tag. A question was raised about having a POST as opposed to a GET request. It seemed like a short enough question, but after looking into it for about 15 minutes, I figured it was worth it to type something up.

To recap, when I first was doing web stuff in high school, I never got what the difference between POST and GET were. It seemed like the author of my HTML book then didn’t really get it either. However, it wasn’t until DHH started raving about REST as a web service did I get it. in short, when a user submits a form, GET puts the form contents in the url, and POST puts the form contents in the body of the HTTP request. The implication (apparently related to REST) is that you’d only use GET for server requests that didn’t change content on the server–the request had no side-effects. That way, the same URL would refer to the same resource, time after time. Alternatively, POST is for server requests that change the content of the server. And that’s why we care.

I find that reading documentation helps. So I first looked at the browser source for a link_to_remote() call and it’s pretty clear it uses Form.serialize() to convert the form, and then sends it along using Ajax.Request(). The question is, does it send via POST or GET?

According to the documentation for link_to_remote(), it sends via POST by default.

The method used is by default POST. You can also specify GET or you can simulate PUT or DELETE over POST. All specified with options[:method]


  link_to_remote "Destroy", :url => person_url(:id => person), :method => :delete

I also checked it out with a quick check inside a controller with a breakpoint to see if a link_to_remote() call really sends a post. I put a breakpoint inside a controller method called by link_to_remote:

$ ./script/breakpointer 
Executing break point at ./script/../config/../app/controllers/friend_controller.rb:60 in `test_ajax'
irb(#):002:0> request.get?
=> false
=> true

As for a regular form_for() call, simply use the :method option. I think by default it also uses POST.

Erlang and neural networks, part I

So it’s been a whole week since my interesting post about the OwnershipFilter. I was investigating several things all at once, all the while wrestling with internal motivation (another post, at another time). In any case, I thought I’d blog about it entirely when I had something full and concrete to show you. However, if it takes me that long, I might as well blog about it as I go along–and it might be more fun for you anyway.

Trace it back

It started with an article about how the free lunch is over for software engineers that a friend sent to me about two years ago. It basically stated that developers have been riding on the wave of Moore’s law to save their butts, and it’s not going to last forever. In addition, it’s been known for a while that chip manufacturers are moving towards multi-core processors to increase performance. If developers are going to take advantage of hardware like they have been, they’re going to have to learn how to program concurrent programs.

The problem is, programmers suck at it. It’s well known that concurrent programming, as it stands, is not easy for humans. Even Tim Sweeney, the guy that architected the Unreal Engine (no mere programming mortal), thought it was hard. It was when I started looking beyond threads as a concurrency abstraction that I tripped over a programming language developed specifically for concurrency.

Yet Another Programming Language

A friend of mine, who is a teacher (i.e. not a programmer), recently asked me, “Why learn more than one programming language?” Ahh, little did she know that programming languages inspire what verges on religious debates between programmers. My short answer was, “Each tool is better at one task than another.” I was looking for another language that might do concurrency better.

I had always thought that I should learn more about functional programming. It seemed like an odd beast to me, especially since you don’t change state through side-effects. “How do you get anything done?” It’s kinda like when you first learned that you don’t need GOTO, and subsequently, when you learned that FOR loops suck.

And yet, I never really found a need or a small project I could do with functional programming that might prove to be satisfying. It was only due to the search for better concurrency abstractions that I ran across Erlang, a functional programming language that is used explicitly because it’s good at concurrency. In fact, it’s pretty much the only one out there that touts concurrency as its strength.

It uses the actor model, where processes share no data and just pass messages around. Because there’s nothing shared, there’s no issue of synchronization or deadlocks. While not as sexy-sounding as futures or software transactional memory, the actor model falls nicely along the lines of complex and emergent systems–systems that have locally interacting parts with a global emergent behavior. Hrm…could one of these systems be good for a small side project to do in Erlang?

How Gestalt, Mr. Brain

Artificial neural networks seemed to be the perfect thing actually. A quick, quick diversion into what they are.

A feed-forward artificial neural network is basically a network of perceptrons that can be trained to classify (ie. recognize) patterns. You give the network a pattern as an input, it can tell you the classification of that input as an output.

You can think of a perceptron much like a neuron in your brain, where it has lots of inputs and one output. It’s connected to other perceptrons through these inputs and outputs and there are weights attached to the input connections. If there is a certain type pattern of input, and it passes a threshold, the perceptron ‘fires’ (i.e. outputs a value). This in turn might activate other perceptrons.

Even simpler, a perceptron is modeled as a function that takes a vector x as an input and outputs a number y. All it does is take the dot product of the input vector x with weights vector w, and pass it through a non-linear and continuous thresholding function, usually a sigmoid function. And you connect them up in layers, and you get an artificial neural network, that can learn to recognizeclassify patterns if you train it with examples.

It has to learn patterns by adjusting the weights between perceptrons in the network after each training example, and you tell it how wrong it was in recognizing the pattern. It does this by an algorithm called back propagation. It’s the same page I lifted all these pictures from. I put all their pictures in an animated gif to illustrate (click on it to watch):

In the first part, the example propagates forward to an output. Then it propagates back the error. Lastly, it propagates forward the adjusted weights from the calculated error.

I think the shoe fits, sir

Why would this be a good fit as a subject to play with Erlang? Well, if you’ll notice, each perceptron only takes input from its neighboring perceptrons, and only outputs to its neighbors. This is very much in line with the actor model of concurrency. Each process would be a perceptron, and would act as an autonomous agent that only interacts with other processes it comes into contact with–in this case, only other perceptrons it’s connected to.

In addition, you’ll also notice that in the animation, the perceptron values are calculated neuron by neuron. In a concurrent system, there’s no reason to do this! You can actually do the calculation layer by layer, since the calculations of any individual perceptron only comes from the outputs of the perceptrons in the layers before it. Therefore, all outputs for perceptrons in a layer can be calculated in parallel.

Notice, however, that layers need to be calculated serially. I had originally thought that with the learning process propagating back and forth, maybe it could be pipelined. On closer examination, however, the best one can do is to feed-forward the next input one layer behind the adjusting of weights, to make the learning process go faster.

So I’m going to give it a shot on the side, and document it along the way, since a quick search on google revealed that though people talked about it, no one’s ever really given some snippets on it. Wish me luck!

Erlang and Neural Networks Part I
Erlang and Neural Networks Part II
Erlang and Neural Networks Part III

Stream Copy YouTube, Revver, Etc.

For lots of times in the past, I was always annoyed that I couldn’t save content offline, and always had to view it online–especially when there’s no guarantee that the content will stay there. So, I was a little bemused, but not altogether surprised, that you can save streaming copies of videos using Ruby. The source code is fairly short…maybe 20 lines or so. And while I haven’t tried it out, I’ve actually been able to read it through, and it’s nice to be able to read the works for people far better than me, even if it’s just table scraps.

Well, if you want to try it out, the balloon is here. A balloon is a web page that has ruby embedded in it, so that you can run it in ruby. Just make sure you have ruby installed on your system, and follow the instructions on the balloon.