All about Mobile, Web, Technology, Politics, Security, E-Commerce and Living in the Philippines.

Thursday, May 29, 2008

Twitter Achitecture

This is my two cents on improving Twitter Architecture:

I work for a mobile service provider in PH and a bill payment company in AU. I feel empathy towards twitter because I've been on the situation before since I also work on a high-availability and high-traffic systems as well.

Although most of these suggestion is based on  my work for on  machine-to-machine messaging, I think the following suggestion would work for Twitter.

DB Sharding: Current twits (1-4 weeks old) has higher probability to be access via web/API than older twits. Use Sharding by date (eg:monthly) so it is possible to allocate old twits to a lesser priority (less likely to take up memory as DB index). The project HSCALE has an excellent presentation slide explaining how sharding is implemented.

Decentralize: Twitter is a little bit like blogs but a lot simple, however its like a messaging system too. Therefore proposed solutions that treats Twitter as a messaging system will not completely solve the problem. Like a patient in a hospital, assuming that all symptoms observed are related to a single disease are often fatal.

Twitter is unlike IMs which basically has a 1-to-1 or many relation in terms of message delivery and that the largest group of recipient will always be finite. Twitter on the other hand suffers from 1-to-infinite relation! Because of this it is crucial that all data reads must be decentralized, which is the same way how blogging system works.

API on reading data for public accounts should be done via RSS only such that it can be redistributed evenly by aggregators likew FeedBurner. Of course there is a sacrifice, the system will lose its just-in-time message delivery.

Anyway, even Google can't crawl the net and display changes to search results that fast; why should twitter even try?

Lastly, Data localization: Take advantage of emerging technology such are Googe Gears which allows data persistent to be localized on the user's computer. This greatly reduces the need to transfer data, lessen impact caused by DB reads and boost usability by improving UI responsiveness.

Friday, May 16, 2008

My Twitter Fave: Fun with TwitSnap

The following are my favorite Twitter users I'm following:

introducing @mkeduardo, @aileenapolo, @digitalfilipino, @chattee, @eclair, @ajbatac, @jerryc (the insomiac :D), and @kuting (

Wednesday, May 14, 2008


UPDATE: TwitSnap beta testing is now open to the public, go to to register!

Today I'm releasing a beta-test of TwitSnap!

TwitSnap is a FREE widget that automatically provides Twitter status preview of a user like this: @godie

Notice that I'm using the conventional reply-to prefix for Twitter and nothing else! The widget will automatically parse your page and look for the pattern, convert it into a link to user's profile on Twitter and then provides a preview of the user's most current status via a pop-up Dhtml box.

TwitSnap is built using Google Apps Engine (Python runtime), some really wicked Regular Expressions (REGEX), insane DHTML JavaScripts and of course using the Twitter API.

There are still some few things needed to be ironed out:

  • Google Apps Engine where the app is hosted has a bandwidth limit; dear Google, care to donate more bandwidth to me? :D

  • Twitter limits the number of requests per IP; I'm still trying to contact them on having the service get increased limit.

  • I'm just piggybacking on my neighbors' WIFI; Ok this isn't really a good reason but I wish to thank her for sharing the Interner access which allowed me to work at at.

Note: TwitSnap is not related to, this is work in progress. Use at your own risk! we do not guarantee that anything will work. It may even break your site rendering so use with caution.

TwitSnap has been tested on only FireFox, Opera and IE.

Sunday, May 11, 2008

Blog.Gears Standalone

Im a fan of Google since most of their services truly enhances the way we live in this technological era. A few months back i was looking for a Blogger client and most preferably with an offline and sync capabilities. However there is none available for windows and some costs a lot without even fully supporting Blogger.

Then I stumbled into Blog.Gears which is an offline Blogger tool, unfortunately this tool was only developed to demonstrate the capability of Google Gears. I also found out that it doesn't run as a standalone desktop application. But since i really wanted to be able to blog offline I ripped the sample app from the Google site and hosted it in my local web server. This for sometime has been my source of blogging joy and I wish to share this to all Blogger users also. However, the main problem is the requirement to install a web server such as IIS or Apache which makes it cumbersome or even technically difficult for some.

So I made some research for the lightest HTTP server that will run on windows and found sHttpd. Finally I made a batch file launcher and edited the application a bit to include some information on how to effectively use the app.

Now you can download Blog.Gears Standalone Edition and enjoy offline blogging.

Note: The archive is in Zip format and the password is

Related Links