Segmentation of social news sites

Giles Bowkett: Summon Monsters? Open The Door? Heal? Or Die?

I have to admit, I almost stopped reading after the first couple paragraphs justifying himself being jerk-ish, but he does have a healthy dose of good points towards the middle.

The underlying assumption of ‘wisdom of the crowds’ is that people make independent decisions, and they have the same amount of time to do it. Neither are true in social news sites. The former being untrue because you can up vote stories that are already on the front page. That just makes it into a positive feedback system that blows up and amplifies small signals. It’s ok when the community is small, but as it gets larger, the more likely that noise will make it.

The second point I didn’t think about until Giles pointed out explicitly–that since votes come for free, that people that spend their time on the sites are the ones that influence it the most.

Combine the two effects, you have a recipe for amplification of noise. The problem is, you need the amplification mechanisms like up voting on the front page in place when the site is small to grow it, and then when it reaches a certain size, the mechanics of social sites need to change (to what, none of us exactly figured it out yet) to protect the users from themselves.

I’m venturing to guess personalization and fuzzy segmentation to be one solution. As Paul Buchheit mentioned earlier about how twitterers hardly get any spam, it’s because if anyone’s saying stuff you don’t want to hear, you can just unfollow them. Twitter works in this regard because there is a built-in small world network with a relatively low transmission rate between nodes (as opposed to facebook which has a small world network, but high transmission rates of information between nodes…which results in lots of unwanted invitations to bite zombies and vampires). Social sites like Digg, reddit, and hacker news, don’t really have a network. It’s just one single “place”, where what happens on it affects everyone, and small perturbations get amplified.

However, I don’t think such a strategy would work well in the beginning. The very thing that helps a small community in the beginning hurts a larger community, and the very thing that would protect a larger community from itself would stunt the growth of a new smaller one.

I think this would be an interesting topic and ripe for research. It actually reminds me of ant colonies, where younger ant colonies will act like teenagers, taking more risks, focus on growing, and experimenting. Older ant colonies are more about taking less risks, maintaining the brood, and surviving. There’s some sort of decentralized mechanism that kicks in for ant colonies to do that, or maybe once they reach a certain size. I think looking into the literature for that might yield some clues into how to design community sites so that they can grow in the beginning, and not implode when they get bigger.


Gotchas of internal iFrame facebook apps and external web apps using Facebooker gem

A while back, I added mobtropolis to facebook as an internal app. I decided to go with using FBML because there was more support in the how-tos about how to use it, and it looked like tighter look and feel and integration.

However, unlike many facebook apps, Mobtropolis also exists as a stand-alone external web app. This decidedly made things a little bit hairier, and I had to write a custom mime-response filter to be able to tell whether a call was coming from a web client (HTML), or as an internal facebook app (FBML), in order to authenticate correctly. I also ended up having to write some custom testing methods for it as well.

Then I revamped the layout of mobtropolis.

It’s major suckage to have to maintain two separate views, so I decided to go with an iFrame with the internal facebook app. It took a bit of work to convert it to use iFrames, because authentication gets a little bit more complicated. However, it’s something that I only have to deal with once. Subsequent changes to the layout won’t affect it as much.

In retrospect, I should have went with using an iFrame from the beginning, though, at the time, mobtropolis was fairly ugly. This is what people call “judgement“, and I made the mistake and it cost me about three weeks. The thing is, you just make the best decision you can at the time, and make sure you can change directions easily.

There were a couple gotchas when using iFrames.

  1. Double facebook frames on redirect to install page.
  2. External app’s layout is wider than iFrame
  3. Facebook only sends fb params on the first call to your app

Hopefully, I’ll save you some time, to whomever’s looking for this info.

1) Double facebook frames

When you use ensure_application_is_installed_by_facebook_user or ensure_authenticated_to_facebook, it will automatically reroute the user to an install page if he didn’t install your application. Problem is, it assumes that you’re not in an iFrame. It ends up that you can override application_is_not_installed_by_facebook_user in your controllers.

def application_is_not_installed_by_facebook_user
redirect_to add_internal_facebook_app_url

Where add_internal_facebook_app_url is an action in a controller (say, my_controller), that renders javascript to change the location of the top frame.

def add_internal_facebook_app
render :layout => false, :inline => %Q{
top.location.href = ""

You have to make sure you connect it as a route in order to redirect it like I did in the overridden application_is_not_installed_by_facebook_user(), in routes.rb under config/

:controller => "my_controller",
:action => "add_internal_facebook_app")

2) External app is wider than iFrame

I think there is a way to resize the Facebook iFrame, but I didn’t find out about it after I did this. By default, the Facebook iFrame “smartsizes” itself, to fill out rest of the page.

First, I created a stylesheet called fb_internal_layout.css, that had extra stylings that squeezed the interface in a 446px wide iFrame. Then I included it in the headers of my layouts as:

Make sure you include titles in the link, so that you can actually switch it out.

Then we use javascript to turn on or off this alternate stylesheet depending on whether we’re in an iframe or not. You can use something like what’s described in A List Apart’s article on alternate stylesheets to switch out stylesheets.

To detect if I was in an iFrame, I simply checked whether ( == frames.self). If it was, I turned on the alternate stylesheet.

3) Facebook only sends fb params on the first call to your app

This is actually not a problem if you use FBML. This is also not a problem if you’re using iFrames, and you require a user to install your facebook app if they want to see what’s on it.

However, even though this is how a lot of facebook apps operate, I don’t think this is very user friendly. The user has no way to judge whether they want to install your app or not if they can’t even sample it. I would rather have a user add an app because they want to, rather than getting people that add it, but then remove it shortly after. This not only gives you an inaccurate indication of how many people really want to use your app, but also annoys the hell out of them.

But making some pages of an iFrame app to be public is a bit tricky. Only the first click into your facebook app is there fb_params in the request. Every subsequent click by a user is in your iFrame, so looks as if the user is actually on the external webpage.

There are a couple solutions, but I ended up storing session state that the user made a request from an internal app before. You can’t override params on subsequent requests, so using old fb_params to authenticate is difficult at best. Using the flag that a user made a request before, this session is likely to be coming from an internal facebook app. When it comes upon a private page, it should be redirected to install mobtropolis, using 1) detailed above. This is not a perfect solution, but it covers all cases correctly.

This, however, doesn’t account for the instance where a user that already installed. In that particular case, I just went ahead an got a facebook session on every first request to the facebook app.

Hope that helped, and I hope never to have to mess with this sort of stuff again, and that you don’t either. More interesting posts in the future. Tip!

MIME responder filter for Rails

I didn’t think I had to do this, but I ended up writing a filter that acts like a switch statement for different MIME types. Let me explain. Normally, in Rails, you can respond to different requests for different content with something like this:

class PostController < ActionController::Base
def list
@posts = Post.find(:all)
respond_to do |format|
format.xml { render :xml => @people.to_xml }

When you have something like this, the browser (or whatever client) can ask for different MIME types. Here, we can return html to a browser, xml to maybe a data importer, and fbml to facebook.

I spent last week integrating Mobtropolis with facebook. Mobtropolis doesn’t require a facebook account to use it, so like other websites, it has its own authentication mechanism, something like:

class PostController < ActionController::Base
before_filter :website_authenticate_filter, :except => [:index, :list]

When I started using facebooker library, it already came with an authentication before_filter. That means we have two authentication filters, one native, and one for facebook. Mobtropolis users don’t have to be in facebook to use it, and facebookers don’t have to sign up again in mobtropolis to use it.

However, since before_filters are executed in succession, it leads to a case where the facebook authentication would be called if html was requested, and vice versa. The alternative was to take apart both authentication filters, and create a monolithic filter to handle the two different cases. Instead, I did this:

class PostController < ActionController::Base
before_respond_to_filter :except => [ :index, :list ] do |format|
format.html :website_authentication_filter
format.fbml :facebook_authentication_filter

That way, I didn’t have to mix together the guts of each authentication filter, and it solved the problem of the wrong authentication filter being run. You can also use it like:

class PostController < ActionController::Base
before_responds_to_filter :only => :home do |format|
format.html do |controller|
return if controller.logged_in?
controller.send(:redirect_to, :controller => :home)
format.fbml :ensure_application_is_installed_by_facebook_user

By the way, I tried to alter the filter_chain as a request came in. Filter chains are copied and passed around the filters, so you can’t write a filter that alters the filter chain. So don’t waste your time crawling around in the guts of Rails to do this like I did. It’s just as well, as that’d be a nightmare to maintain.

It does have some weaknesses though. You can only assign the filters to the same set of :except and :only options in the filters.

It ended up the code for this sort of magic was fairly easy. I’m not sure if there’s an easier way to do what I wanted, but I’ll see if Rails core people would find it useful (or not). In the meanwhile, for those of you Rubyists that have written plugins before that want to play with it. As with the usual mumbo jumbo, it’s provided as is, I’m not maintaining it, and do whatever you want with it:

module Threecglabs
module Filters

# MimeResponderFilter
module MimeResponderFilter

def self.included(mod)

# Filters can respond to different mime types, so that you can use
# different filters depending on which mime type is being requested
# before_responds_to_filter :except => [:login, :signup, :forgot, :invite_request, :profile] do |format|
# format.html :authentication_filter
# format.fbml :ensure_application_is_installed_by_facebook_user
# end
# This way, one can take the appropriate actions in setting up authentication
# from different mime types, and still separate the implemenation of the different
# kinds of implementations
# The formats also take blocks, like regular filters
# before_responds_to_filter :only => :home do |format|
# format.html do |controller|
# return if controller.logged_in?
# controller.send(:redirect_to, :controller => :home)
# end
# format.fbml :ensure_application_is_installed_by_facebook_user
# end
# NOTE: an :all format defaults to :html, therefore, a format.html is required
module ClassMethods
def before_respond_to_filter(options = {}, &block)
before_filter, options

# This is a call that implements a MIME responder filter
class MimeResponderFilter#:nodoc:
attr_reader :filters

def initialize(&block)
@filters = {}

def filter(controller)
filter = @filters[controller.request.format.to_sym] || @filters[:html]
if filter.kind_of?(Proc)

# implements the "format.#{mime_type}" part of the filter
def method_missing(mime_type, method_name = nil, &block)
if block_given?
@filters[mime_type.to_sym] = block
@filters[mime_type.to_sym] = method_name.to_sym




Review of Facebook app building experience and where things might go

I’ve been missing in action for a while, because during the first week of August, I finally got around to figuring out how to build a facebook app. I’m involved with a non-profit organization that wished to create an alumni database. I had written a prototype for them the year before in a week, but I never put it up. I didn’t get the help or traction that I had hoped for, so I delayed working on it until this past summer, when they had another summer conference.

It’s a good thing too, because facebook didn’t release their full API until around May. The app is a good fit for what it needed to do, since it allows alumni to find each other. I hacked it out in 4 days, which was both good and bad. I had to bypass a lot of the discipline and good habits I cultivated on my own projects in order to whip it up.

The hardest thing about facebook apps is simply figuring out how the whole thing hangs together. When you don’t know the thing is put together, filling out the long form to setup your facebook app, becomes difficult as nothing makes sense.

There are two major parts to a facebook app. One is a profile box, which is the draggable box you see in people’s profiles. And the second is the canvas page, which is a kind of “home page” for your app, if you will. You application is hosted somewhere else outside of facebook. The API just lets you create an interface in facebook. It’s my recommendation that you read this page on architecture in the facebook wiki before getting started on any of the “Getting Started” pages.

As you can see from the ASCII diagrams in the architecture page, your application’s canvas page is called by proxy through facebook servers. That means every time someone visits your canvas page through facebook, facebook will pull your application’s canvas page and display it. Conversely, the profile box is data that is pushed to the facebook servers. This makes sense in two faces of the same coin. Facebook has a LOT of users, and chances are, you can’t scale as facebook has. Having lots of people bang on your server out the door would wreck havoc on you. Secondly, facebook doesn’t want its profile load times to depend on 3rd party servers. If it has all the information pushed to it, facebook can quickly serve profile pages without having to rely on the latency of its 3rd party applications.

I won’t go through how to make a facebook app. Plenty of places already do that well, especially this one for Ruby. However, to give my review, I will say that I give it 3 out of 5 stars. “A” for effort, but there’s more improvement to be had down the line.

While the API is a step in the right direction, coding for it has felt clunky at best. This is mostly due to documentation being a bit all over the place, but especially the lack of a test environment. There is no way to set a production, test, or development environment for the same application. The only way is to simply create another application as a “test”, and point it to your development machine. Then in your code, it has to switch between the development API key and the production API key. In Rails, you’d use the RAILS_ENV global variable to determine which environment you were in. As a result, there’s quite a bit of setup to do before you can start hacking away.

Facebook also provides FBML, a reduced HTML, specific for facebook elements. Some of them are quite handy, such as a friend finder, and it gives you an immediate look and feel that blend in with the rest of facebook. However, debugging it isn’t much fun at all. Since you can’t set environments, facebook assumes all apps are production, and therefore will not show errors and stack traces. Therefore, one has to tread softly, or cut and paste back and forth to their tools.

That’s right. Tools. Facebook does provide tools to test out your FBML and API calls. But it would be much nicer if it were integrated in a development environment for the application.

All, in all, it’s not been too bad of an experience, outside of wrestling with the setup and FBML, but there’s a lot of improvement to be had. While many have said that Facebook is a walled garden like AOL was in the 1990’s, I currently don’t see much of a decentralized social network solution that addresses privacy that is popular amongst developers.

The only thing that is a remotely good candidate is the combination of OpenID with Microformats like XFN. But I think it’s going to take a demonstration from another successful startup to get more developers on this bandwagon. OpenID is also not yet ubiquitous as a concept in the minds of developers, let alone the common web user. Once this happens, it will be much much easier for social applications to personalize a user’s experience right out of the box.

And this isn’t personalization based on where your stats, but personalization based on what your other friends have done in the very same application. The web app won’t know anything about where you live, or where your friends live, but if it’s a book store, it will know everything about what books you like, and what books your friends like, and be able to draw conclusions to better serve your experience based on that.

I hope that is an indication of the things to come. Apps will know more about a user’s habits as related to their function, and yet not know who this user is to protect privacy. This will be especially useful with mobile devices, since we carry them with us. Any mobile application that knows if you come into contact with your friends in real time will have a tremendous utility value in producing information useful in a social context. We still have to wait, however. The mobile industry is still not as open as we’d like for open development on mobile devices as a platform.

As an aside, I imagine that the reams of data (anonymous, of course) would be a great boon to social scientists. We might see a revolution in that field, especially if it’s combined with decentralized computing concepts.