Archive for the ‘Uncategorized’ Category
The Four Hour Web App: Venn’d
I am a huge proponent of building quick web prototypes. Launching a quick and dirty tool helps guage interest in a new product or service, exposes you to challenges and obstacles you may not have foreseen, and provides real data on usage patterns and user feedback which are invaluable during the conception and design process.
With that background: Wednesday night I saw this tweet from Jason Fried, founder and partner at 37 Signals:
For a not-yet-launched project, I have been working with some new functions in the Twitter API, only days old, that simplify the process of polling for a user’s friends and followers. I thought it would be fun and educational to launch a super-quick using the same toolset, and so Venn’d was born. Venn’d is a dead-simple tool: given two Twitter users, it calculates the overlap between each of their collections of friends and followers, and presents the results using a Venn diagram. Here’s a behind-the-scenes look at the services and tools I used to build it.
The core functionality of Venn’d is driven by Twitter’s API, specifically the calls to request a user’s friends and followers.
For a quick, “free”, scalable hosting platform I chose Google’s AppEngine. I put free in quotes because eventually you’ll be able to purchase more AppEngine computing resources if you go over the free quotas, but that functionality hasn’t been rolled out yet. Google says that the platform will remain free for applications requiring less than 500 megabytes of storage and generating fewer than 5 million monthly pageviews. Beyond those levels, the expected pricing is:
- $0.10 – $0.12 per CPU core-hour
- $0.15 – $0.18 per GB-month of storage
- $0.11 – $0.13 per GB outgoing bandwidth
- $0.09 – $0.11 per GB incoming bandwidth
The current functionality of Venn’d only required a few of AppEngine’s service APIs: DataStore to record the details of each request (except for Twitter username and password information) and URL Fetch to access the Twitter API. I used my own modified version of a Python REST framework to streamline the code under the hood (can’t find the original source I used at the moment, I’ll update this post if I track it down later). My knowledge of Python is only just passable; the nuts and bolts of Python and AppEngine would’ve been faster for someone who really knew their stuff.
I chose jQuery as the JavaScript/AJAX framework. I typically use Prototype, because of its tight integration into Rails, and thought this would be a good opportunity to learn more about an alternate tool. The jQuery form plugin and validation plugin made it easy to add AJAX validation and submission to the main form. Instead of hosting jQuery locally within the application, I referenced the copy of the library that Google makes accessible through their AJAX Libraries API. Because a signficant number of other sites around the web do the same, the odds that a user will already have a cached version in their browser goes up, improving page load times and performance.
To generate the Venn diagrams, I used the Google Charts API. This made it quick and easy to visualize the overlap between each user’s friends and followers. The designer in me wanted more control over the look of the charts, so I spent a little time looking into generating custom charts from scratch. Turns out it would be a huge pain, and not worth the effort. One of AppEngine’s restrictions is that you have no access the filesystem of the server running your code. Essentially all of the available image generation libraries available for Python presume such access, so you can’t use them. Google recently released an Image API for AppEngine, but it can only handle basic rotate/resize/crop type manipulations, not generating an image from scratch. There are a lot of forum threads clamoring for better image support, so this may be functionality Google provides in the future. I briefly considered using a browser-based solution like SVG or CSS with transparent images, but thought both would be too much effort and have browser compatibility issues. Using an API is always a compromise between the speed and convenience of using an off-the-shelf solution and the power and customization of writing your own code. For a quick and dirty tool like Venn’d, using an existing API was the way to go.
As a small nod towards making the app more viral, I added a “tweet this” link to each generated diagram to make it easy for a user to share the results on Twitter if they wanted to. To deal with Twitter’s 140 character cap, I used the bit.ly API to generate a shortened version of the permalink to each new page.
I make it a point to add Google Analytics from the get go to every site I build. It might not be necessary at first, but having good data on how your application is being used is invaluable as your traffic grows. And check it out! 40 whole pageviews! :)
Finally, a few notes on design. I didn’t want to invest much time or energy into the visual look of the app. I grabbed a royalty-free pattern image to use as the background. For a color scheme, I sampled a few of the primary colors from twitter.com itself using the ColorZilla Firefox plugin‘s eyedropper tool. I test-drove a few CSS typography schemes using Typechart and chose one I liked.
That’s it! The “finished” product: Venn’d
Crowdsourcing, Attention and Productivity
The reigning currency of the internet is attention. Bands posting free songs on MySpace, bloggers churning out a steady stream of screeds, and teenagers pouring their hearts out to their webcams on YouTube all crave attention from their peers or the wider internet community. Yet as critical as this dynamic is to today’s web, I’m aware of almost no research which quantifies its inner workings. So I found this recent paper from HP’s Social Computing Lab very interesting: Crowdsourcing, Attention and Productivity.
In the paper, Bernardo Huberman and his co-authors set out to measure how the popularity of someone’s web content influences their decision to produce further content. The researchers used a dataset of users who had contributed videos to YouTube. Their first insight is that user contributions are “bursty” — on average, users posted nothing in 66% of the two-week time periods the calendar was sliced into. More importantly, there was strong evidence that attention, in the form of video views, encourages users to produce more videos. The more attention a user’s videos received in one period, the more likely they more to upload more video in the next (see the paper for a full description of the linear regression model used). On the flip side, users who stopped uploading content tended to do so after a steady decline in the views they received.
How does this relate to strategy? Let’s say that YouTube could choose between two scenarios: 500 user-contributed videos with 100,000 views each, or 5,000 videos with 10,000 views each. Which would they prefer? Both scenarios deliver 50 million pageviews, roughly what the site currently gets per month. In the first scenario, only 500 users are receiving the attention that will encourage them to continue creating content, while in the latter that number is ten times higher. The latter, larger pool of users will be the engine of growth, creating more content and adding value to the site.
This isn’t just a thought experiment. Websites have powerful user experience levers with which to influence the shape of their communities. YouTube’s home page could be a simple list of the top 10 most-watched videos this month, reinforcing those videos already popular and creating a site with a few “blockbusters” and a great many more videos languishing in obscurity. Alternatively, the home page could list 10 videos being watched right now, a few videos selected at random, and maybe a selection of videos whose popularity is rising quickly or already strong among a given subset of the community. This would be a more egalitarian YouTube, where niche content had a greater chance to find its audience and grow in popularity.
This flatter, more egalitarian YouTube strategy finds support in the economics of web audiences as well. Niche audiences, though small, are worth more per capita to advertisers than the monolithic audiences of mass media because their interests are more specific and can be matched more precisely to relevant advertising. (Online advertising wonks: feel free to push back on this argument.) The ad buying world may not have fully adapted to this view yet, but a YouTube comprised of thousands of micro-communities has significantly more value than one with a single undifferentiated audience. The Social Computing Lab‘s research provides quantitative support necessary to cultivate these communities.
Microapps for Fun and Profit
The future of the web is tiny. We are moving from an era of big, monolithic web sites to small web services being stitched together endlessly in new, innovative ways — small pieces, loosely joined, to use David Weinberger’s phrase. These tiny pieces of the web can be thought of as “microapps“, and MicroApps.org describes the philosophy in greater detail:
MicroApps are small REST applications that are designed from the ground up to be integrated with other applications. Usually, they are not directly useful on their own, but must be integrated into other applications (this is what differentiates a MicroApp from a regular REST application).
And further…
The core idea of Microapps is basically using the web (and REST specifically) as a component architecture to build applications. A microapp is a small application with a very tight focus that can be integrated with other microapps or other web applications via HTTP and a common data format (usually XML, JSON, or RDF).
Don’t worry about the technical details about REST architectures and data formats; the key point is that microapps are small pieces of functionally designed to be put together into larger applications. But if you browse the list of existing microapps, something strange becomes apparent. The vast majority of these microapps are not hosted services, but rather Plain Old Software, which you download, install, configure and manage yourself. These tools would be far more valuable if made available as public, hosted web services. For one, the potential audience would be vastly larger, expanding from the relatively modest set of developers with access to their own web servers running Python, to, well, just about any web developer at all. Secondly, a lot of collective effort could be avoided if each new user of a microapp didn’t have to install and manage the software separately. So what’s the problem here?
Erik Kastner gave a presentation at RailsConf 2008 called Microapps for Fun and Profit. He defines three categories of motivation for someone to create a microapp: “fun” (play, learning), “profit” (AdSense, donations, sponsorship), and “better than money” (reputation, your personal brand, experience). Erik also makes reference to “microprofit” (seen in the snip of one of his slides above), and this hints at the barriers to turning microapps into fully public, hosted services.
Coming up next: why there aren’t more microapps, and what we can do about it.
Sneak Peak 3: TwipJar
Twitter-powered social micropayments platform.
Amazon Start-Up Tour
I took a break this aftrenoon from the Web 2.0 Expo for the Amazon Web Services Start-Up Tour. It was a cool event highlighting startups which are building their businesses on top of AWS, as well as some good talks by VCs about what they’re looking for in today’s startup space. Highlights included Adaptive Blue founder Alex Iskold describing some unique ways his company is using the services and a talk by Nick Beim of Matrix Partners about doing deep analyses of your customer microeconomics.
Amazon’s Web Services are a hugely important addition to the web 2.0 space, and one who’s possibilities are only starting to be fully capitalized upon. AWS Evangelist Mike Culver described how using Amazon Web Services can eliminate the “undifferentiated heavy lifting” that stands between idea and execution for a startup. I’ve made use of S3, SimpleDB, and SQS for my recent web projects, and am working on incorporating the Flexible Payment Service (FPS) into another prototype now. Taking advantage of these tools helps startups focus on the areas where they really add value, while leveraging Amazon’s infrastructure for the rest.