Segmentation of social news sites

Giles Bowkett: Summon Monsters? Open The Door? Heal? Or Die?

I have to admit, I almost stopped reading after the first couple paragraphs justifying himself being jerk-ish, but he does have a healthy dose of good points towards the middle.

The underlying assumption of ‘wisdom of the crowds’ is that people make independent decisions, and they have the same amount of time to do it. Neither are true in social news sites. The former being untrue because you can up vote stories that are already on the front page. That just makes it into a positive feedback system that blows up and amplifies small signals. It’s ok when the community is small, but as it gets larger, the more likely that noise will make it.

The second point I didn’t think about until Giles pointed out explicitly–that since votes come for free, that people that spend their time on the sites are the ones that influence it the most.

Combine the two effects, you have a recipe for amplification of noise. The problem is, you need the amplification mechanisms like up voting on the front page in place when the site is small to grow it, and then when it reaches a certain size, the mechanics of social sites need to change (to what, none of us exactly figured it out yet) to protect the users from themselves.

I’m venturing to guess personalization and fuzzy segmentation to be one solution. As Paul Buchheit mentioned earlier about how twitterers hardly get any spam, it’s because if anyone’s saying stuff you don’t want to hear, you can just unfollow them. Twitter works in this regard because there is a built-in small world network with a relatively low transmission rate between nodes (as opposed to facebook which has a small world network, but high transmission rates of information between nodes…which results in lots of unwanted invitations to bite zombies and vampires). Social sites like Digg, reddit, and hacker news, don’t really have a network. It’s just one single “place”, where what happens on it affects everyone, and small perturbations get amplified.

However, I don’t think such a strategy would work well in the beginning. The very thing that helps a small community in the beginning hurts a larger community, and the very thing that would protect a larger community from itself would stunt the growth of a new smaller one.

I think this would be an interesting topic and ripe for research. It actually reminds me of ant colonies, where younger ant colonies will act like teenagers, taking more risks, focus on growing, and experimenting. Older ant colonies are more about taking less risks, maintaining the brood, and surviving. There’s some sort of decentralized mechanism that kicks in for ant colonies to do that, or maybe once they reach a certain size. I think looking into the literature for that might yield some clues into how to design community sites so that they can grow in the beginning, and not implode when they get bigger.


Nerd time – issue 10

Nerd time is just a short mailing list I put out to my ex-coworkers at APL. They’re in the applied engineering research fields, so what’s going on in the web world isn’t well known to them in their daily work, so I fill them in from time to time. If you regularly read techcrunch, proggit, or slashdot, I’m sure you’ve seen these before.

Hey all,

I was going to make this one about new services I use that might not be well known, since there’s nothing terribly interesting going on lately. But after a month of haphazardly collecting interesting things, no particular pattern appeared. Just a hodgepodge of things I found interesting. There’s nothing terribly hard this time. All easy reading, except for the one on APL at the end.

Take screenshots to measure your productivity.
This is something Ian’s been asking for, and I thought he’d like to check it out. No Linux client yet though.

Prof. Randy Pausch’s Last Lecture
This is a CMU prof that is dying of cancer, and he gives a last lecture. You can skip all the intros and extros, as the actual lecture is about an hour. It’s pretty good, and entertaining. I found his lecture of time management to be pretty helpful.

Voice tracking camera
This is one of those “simple” things that you wish you did. Theoretically, it’s pretty easy. You use microphones to do triangulation, to figure out where the voice is coming from. But when you see his setup, he uses seven all around the room–so it might be a bit complicated. It’s a long way from our own two ears.

Commenting engines
Commenting is one of those fundamental aspects of web interaction that gets implemented over and over again, in wikis, in forums, in social apps, in blogs. But with commenting comes a host of problems. Some technical such as spam bots, cross referencing them, keeping the most relevant ones. Some social, such as trolls, scaling a conversation, etc. These two implement that for you, and commenting becomes just a widget. Not a bad idea, especially if they can thread conversations
across different blogs.

Build your own car
I’ve always wanted a hackable Linux based car. Everything from the onboard entertainment system to the safety system. While this isn’t it, it’s a step closer. I think they’ll ship you all the parts you need to build your own car.

Dopplr is a service way to tell your friends, “hey, I’m going to [town], who’s already there, let’s hang out, or I need a place to crash” sort of thing. It’s a social network focused on travelers. I’ve often was somewhere, and found out a friend was there at the same time too, but we didn’t know. It’s still in private beta.

Mozilla Lab’s social network in a browser.
This is an experimental add-on from mozilla that tells you want your friends are doing online. It’s like the news feed in facebook. So any time anyone posts a link, updates their status, etc. you’ll see it. And sending links to people is easy. You just drag it to their photo in the side bar. So instead of me sending nerd time over email,
I might as well blog it or use something like “The Coop”

OAuth is Open Authentication.
I think I posted something about OpenID way back. OpenID is an open way of having uses verify to you they are who they say they are. That way, you don’t have to have a separate login/pass everytime you want to use a new service. OAuth is a way for users to grant permission to a new service for their API. So if you signed up for mobtropolis, and your social network is elsewhere, you’d use OAuth to authorize
Mobtropolis to look up your friends.

XFN microformats and FOAF
This is also part of the effort to open up your social network. Microformats are basically little bits of meta-data inside HTML tags. It’s part of the effort to make the web more semantic. This can be used in conjunction with OAuth to make your social networks portable. We’ll see if people make headway. You can view microformats with the Operator mozilla plugin for firefox. Microformats are actually on quite a few web pages now.

Forth is a stack based programming language. I don’t know as much as I should about it, but it’s mind-expanding. The language lacked conditional branching and loops. But apparently, that’s because you can write your own, not to mention any other weird control structures you can think of. In fact, you can write your own Forth based-PC, its environment, OS, and language in about 2000 lines of code (it is said)

APL — the language
I heard of this language, but never managed to see any code. I can see why. You need a whole other keyboard to program in it. But it is pretty neat. You can write Conway’s Game of Life in one line. I expect this is because it maps well to functions. Neat idea.

Operator for Microformats

I’ve put microformats in the stuff I’ve built before. And to be honest, I wasn’t sure if they were going to take off. The formats were there, but none of the tools to actually use them were around. For a long time, I just used the tails export add-on in firefox. You couldn’t really do anything with it, but you could at least see microformats.

The Operator takes it one step further, and adds actions to microformats. In a way, it’s a god-send, and now I’m wondering why I didn’t write it. I HATE having to cut and paste addresses into google maps. Now, if there’s a microformat of an address on a page now, all I have to do is use Operator–two clicks and I’m there. (As an aside, Humanized Enzo makes it even easier to include maps.)

Microformats have been gaining momentum for a year now, and I think it will be important in the web to come. Not because it’s one-more-thing to have to know, but because the guys over at social network portability are using it as part of their solution to open up the walled gardens of the social network stovepipes in effect today.

Given how we’re all use to using web applications today, it’s as if we’re always ‘reborn’ every time we sign up for a new service. It’s an odd idea to be able to “take your network with you”. If you can do this, especially as mobile devices get more powerful, there can be more and more web applications that can act as mediators in social contexts of the users as they go about the world.

One nice little app would be a name-recaller. At a conference or a party, my mobile phone would detect which other mobile phones are around. It could then query the web application whether I’ve met any of these people before, and what their names are. So when I actually go to shake their hand, I can address them by name. And as we all know, the sweetest sound to a person’s ears is his/her own name.

But that’s still a long ways from now. Probably not in another 4 or 5 years, if not more. However, I’d keep an eye on this area. I’m sure you’d hear more about it soon.

Review of Facebook app building experience and where things might go

I’ve been missing in action for a while, because during the first week of August, I finally got around to figuring out how to build a facebook app. I’m involved with a non-profit organization that wished to create an alumni database. I had written a prototype for them the year before in a week, but I never put it up. I didn’t get the help or traction that I had hoped for, so I delayed working on it until this past summer, when they had another summer conference.

It’s a good thing too, because facebook didn’t release their full API until around May. The app is a good fit for what it needed to do, since it allows alumni to find each other. I hacked it out in 4 days, which was both good and bad. I had to bypass a lot of the discipline and good habits I cultivated on my own projects in order to whip it up.

The hardest thing about facebook apps is simply figuring out how the whole thing hangs together. When you don’t know the thing is put together, filling out the long form to setup your facebook app, becomes difficult as nothing makes sense.

There are two major parts to a facebook app. One is a profile box, which is the draggable box you see in people’s profiles. And the second is the canvas page, which is a kind of “home page” for your app, if you will. You application is hosted somewhere else outside of facebook. The API just lets you create an interface in facebook. It’s my recommendation that you read this page on architecture in the facebook wiki before getting started on any of the “Getting Started” pages.

As you can see from the ASCII diagrams in the architecture page, your application’s canvas page is called by proxy through facebook servers. That means every time someone visits your canvas page through facebook, facebook will pull your application’s canvas page and display it. Conversely, the profile box is data that is pushed to the facebook servers. This makes sense in two faces of the same coin. Facebook has a LOT of users, and chances are, you can’t scale as facebook has. Having lots of people bang on your server out the door would wreck havoc on you. Secondly, facebook doesn’t want its profile load times to depend on 3rd party servers. If it has all the information pushed to it, facebook can quickly serve profile pages without having to rely on the latency of its 3rd party applications.

I won’t go through how to make a facebook app. Plenty of places already do that well, especially this one for Ruby. However, to give my review, I will say that I give it 3 out of 5 stars. “A” for effort, but there’s more improvement to be had down the line.

While the API is a step in the right direction, coding for it has felt clunky at best. This is mostly due to documentation being a bit all over the place, but especially the lack of a test environment. There is no way to set a production, test, or development environment for the same application. The only way is to simply create another application as a “test”, and point it to your development machine. Then in your code, it has to switch between the development API key and the production API key. In Rails, you’d use the RAILS_ENV global variable to determine which environment you were in. As a result, there’s quite a bit of setup to do before you can start hacking away.

Facebook also provides FBML, a reduced HTML, specific for facebook elements. Some of them are quite handy, such as a friend finder, and it gives you an immediate look and feel that blend in with the rest of facebook. However, debugging it isn’t much fun at all. Since you can’t set environments, facebook assumes all apps are production, and therefore will not show errors and stack traces. Therefore, one has to tread softly, or cut and paste back and forth to their tools.

That’s right. Tools. Facebook does provide tools to test out your FBML and API calls. But it would be much nicer if it were integrated in a development environment for the application.

All, in all, it’s not been too bad of an experience, outside of wrestling with the setup and FBML, but there’s a lot of improvement to be had. While many have said that Facebook is a walled garden like AOL was in the 1990’s, I currently don’t see much of a decentralized social network solution that addresses privacy that is popular amongst developers.

The only thing that is a remotely good candidate is the combination of OpenID with Microformats like XFN. But I think it’s going to take a demonstration from another successful startup to get more developers on this bandwagon. OpenID is also not yet ubiquitous as a concept in the minds of developers, let alone the common web user. Once this happens, it will be much much easier for social applications to personalize a user’s experience right out of the box.

And this isn’t personalization based on where your stats, but personalization based on what your other friends have done in the very same application. The web app won’t know anything about where you live, or where your friends live, but if it’s a book store, it will know everything about what books you like, and what books your friends like, and be able to draw conclusions to better serve your experience based on that.

I hope that is an indication of the things to come. Apps will know more about a user’s habits as related to their function, and yet not know who this user is to protect privacy. This will be especially useful with mobile devices, since we carry them with us. Any mobile application that knows if you come into contact with your friends in real time will have a tremendous utility value in producing information useful in a social context. We still have to wait, however. The mobile industry is still not as open as we’d like for open development on mobile devices as a platform.

As an aside, I imagine that the reams of data (anonymous, of course) would be a great boon to social scientists. We might see a revolution in that field, especially if it’s combined with decentralized computing concepts.

I have hope for Facebook being the new Google

Facebook just released facebook marketplace, where its members can sell things, like housing, jobs, or textbooks. Strategically, this makes a lot of sense, since it’s something that’s actually useful to its members, especially if it ties your social network information into what you want to buy and sell. From the looks of it though, it doesn’t do that. But I’m sure someone at Facebook is thinking about it.

Facebook is social networking done right–at least better than any competitors that I’ve seen. On the surface they might all seem the same; there’s a personal profile page, there’s a list of friends, and you can send messages back and forth with each other. However, I think there’s some critical differences.

MySpace has a larger user base, but it is largely seen by its owners as a platform for media advertising. It’s an unsupported assertion, but given its mishmash feature set and large ads, it’s hard to think otherwise.

Friendster was the leader for quite some time, but has since lost the attention of the under 25 demographic (anecdotal evidence). Their mistake was adding things that were technically neat, but ultimately made the site too slow to use. It’s a lot better now, and people are still using it. But based on the features they’ve put out it seems like they are interested in helping people publishing media to a user’s personal network–using blogs, videos, etc. However, no news trickles of them attracting otaku developers, and I’m sure firing the now founder of Renkoo didn’t help win over the hearts and minds of otaku developers.

On the other hand, Facebook is seen by its owners as a platform for technology driven innovation to help keep up social interactions between individuals. I’m not sure when the transition happened, but it was more evident to me after news feeds were released. Now, most people were vehemently opposed to it, but I saw it as two things.

First, it was a feedback mechanism to open up sharing. The more you share about yourself to your friends, the more you appear on their radar, and the more interaction you’ll interact/message them. This seems to be inline with the goal of keeping people talking with each other.

Secondly, it was the basis of publishing personal news without even having to push a button. We all gather news about the world, but beyond CNN, there’s also another type of news we’re interested in–information about what our trusted friends are doing. Blogs lets you publish just by pushing a button. Facebook Mini-feeds lets you publish just by doing what you normally do on Facebook. It’s not inconceivable that in the future, you can also publish from your mobile that you have free time to chill out, and people can just join you to hang out because they saw that you were available in their mini-feed on their mobiles.

Facebook is pulling ahead in terms of their feature offerings because they seem to be able to attract developers that are the otaku of programmers that are willing to innovate something that’s actually useful to their users. Which other social network puts programming puzzles in their mini-feeds? Which other social network has an API? The alacrity in which they deploy features is stunning as well. They implemented twitter pretty easily by listing their status updates. It is in this way that I see them being a ‘new Google’–they are setting themselves up as a hacker’s paradise and attracting otaku programmers that way.

When Zuckerberg held out against getting brought out, he was either being greedy or he had future plans on what he would be able to do with a social network. Most of the press criticized him for being the former, but it’s looking like it’s the latter. As long as Facebook is useful for their users, there’s value in the social network data that can be used by future applications. If they can establish themselves as the standard platform from which all social information about an individual is gathered through their API, this world would be a changed place, just as Google changed the world with its technology.

Vertical markets for social networking sites

Goodreads is a social networking site for book lovers. That really puzzled and surprised me. I know for a fact that vertical (niche) markets are always better to start off with when you’re starting out. It lets you build a community of like-minded people, and it makes you focus your product, which makes it not only easier to build, but makes you design a tighter product.

But the question to me at first impression was, why would people join a vertical market for something that already exists?

Goodreads is a well designed site. It has a core, limited feature, and it’s clean, and it’s easy to add books that you’ve read and write reviews on them. One can also see what your friends are reading and what they’d recommend.

However, the big three social networking sites, Friendster, MySpace, and Facebook all have a section where one can list and name books that interest you. And Amazon, already has both places to read/write reviews for each book, and a recommendation system that works pretty well. In addition, you can actually buy books there.

So what does Goodreads have to offer? Well, for one, at this early stage, it’s fast and responsive, and the mechanisms for actually adding a book is very easy. In addition, it plays on one of the elements of gaming, which is namely, collecting. And that goes along very well with those that read books…they like to have a full bookshelf (almost like trophies on display). There are many games that focus on the collection aspect. Yugi Oh! and Pokemon are the two that spring to mind. Goodreads is like pokemon…for people that like to read.

In addition, I think the book covers help enforce that idea of a collection, so you can see books in your bookshelf. And not only that, you can segment them according to what you’ve read, are currently reading, didn’t finish, etc. That’s something you can’t do with a normal bookshelf.

I won’t say that it’s inevitable that a niche market in an already saturated global market of social networks is going to succeed. But the way Goodreads did it makes sense. In fact, the same mechanism of collection that I outlined above can readily be applied to a social network of fashion geeks.

I own this shirt, or that blouse. I can see what my friends are wearing the next day. I can ask to trade clothes. I can make outfits for myself from my closet, and recommend them to others that have the same pieces of clothing. I can rate clothes and brands, and I can see what the overall trend for different types of clothing are.

I’d be tempted to make it myself, since I usually have an awful time shopping for clothes–I can never remember what I already have. However, 1) I’m not much of a clothes person myself, so I’ll only probably work on the ‘help me choose my clothes’ part, and 2) there are already plenty of niche market fashion sites out there.

What other vertical markets for social networks do you see that would have potential?