Forking Encouraged: Folk Programming, Open Source, and Social Software Development

This is my other presentation from OSCON. I gave this one with all-round lovely guy Yoz Grahame from Linden Lab, and Freebase developer platform czar Jason Douglas. We presented on Friday morning in a really tough slot — there were half a dozen other talks that I wanted to be at! — so attendance was a little light.

There were some upsides though:


So, for those of you who couldn’t make it for whatever reason, I hope this will be useful!

Quick notes:

Yoz first gave his “Folk Programming” talk at a Wiki Wednesday in 2007 and I was there and I thought it was great, so we’ve been talking a lot about the concepts on and off since then.

Basically our presentation here is the 15 minute version of his longer talk, followed by me talking about how the concepts apply to open source software, and then Jason talking about how we (i.e. applied the concepts to our new app development platform to help encourage adoption and a strong developer ecosystem.

Yoz starts out talking about he first learnt to program (on a ZX Spectrum), and how beginning programmers often cargo cult as a way of learning. Cargo cult programming is often stigmatised, but it can be seen as a kind of folk process, like folk music or folk stories, and it’s a good way to get started.

He then gave a bunch of examples of folk programming in action, from an MIT study of children passing around little games on Dreamcast modules, to Yahoo Pipes and Second Life. He identifies five things necessary for a real folk programming environment:

  • Create your own code.
  • View, clone, and modify someone else’s code.
  • Free, always-on, ubiquitous hosting.
  • Huge array of diverse data sources.
  • Code creates new data for use by others.

I then took over and slightly reframed his list for the open source world:

  • Free and Open Source licenses
  • Easy cloning forking
  • Hosted development environments
  • Huge array of data

I looked at some examples including github, Amazon EC2, and Google App Engine, and looked at them to see whether they provided folk programming environments for open source development. What I found is that each provides some elements, but none provides all of them.

When Jason and his team at Freebase were starting work on our hosted app development platform, Acre, we got Yoz in to give a lunchtime talk about Folk Programming, and used a lot of his ideas in developing our platform. Jason walks us through how Acre has BSD licensing, one-click cloning, and interacts with Freebase’s huge store of open data.

Some of the demos he shows include “data games” for Freebase contributors to more quickly add data to Freebase itself, which started off with a single app called Typewriter and quickly evolved into a cluster of apps and eventually a library and toolkit to allow people to quickly build more. (Some were built at our recent Hack Day, for example, including one to easily find CC-licensed images to attach to Freebase topics.)

Another set of apps he showed were clustered around the concept of lists in Freebase, starting with an app to create lists of anything and then showing a recommendation website and an app for TV fans all based on the same underlying data.

Jason closed by talking about some of the challenges we’d faced and future directions for folk computing in the open source world. Challenges included version control and collaboration (because there is a time when you’re no longer experimenting and need to work together on things rather than fork), and the state of in-browser development tools like syntax-highlighting code editors and Firebug-style console debugging for server-side apps.

Questions from the audience included “Have you seen people writing large apps in folk programming environments, in the thousands or tens of thousands of lines range?” to which the answer is “yes” — LambdaMOO and SecondLife being enormous, and some of the Freebase apps being in that linecount range already despite only having properly launched about two weeks ago.

“How do you handle security?” was the second question, which Jason answered at length. Some of the parts of the Acre security model included sandboxing on the server; having each app on its own subdomain to prevent cookie hacks; and everything being sanitised by default to protect against cross-site scripting attacks.