A quick tour of Freebase

I posted a while ago about my new crack habit, Freebase. Freebase is — to my mind — like Wikipedia mashed up with the LambdaMOO class hierarchy. The idea of creating classes and types for real world things — ships, people, breeds of sheep, bones in the human body — rather than abstract and (let’s face it) mind-numbingly dull ones like “result set” or “transaction” or “user agent” is fascinating to me, and the copy of Civilization IV that I recently bought has sat languishing as Freebase seems to be filling that entertainment niche where you start start playing after dinner, and suddenly it’s dawn.

Since I last posted, Freebase has been opened to the public for read access, so you no longer need an invitation to take a look around. So, go take a look! http://freebase.com/ is the place to start.

In this post, I’m going to be giving you a quick guided tour of what’s on Freebase and what you can do with it.

### 1. Watch the video

First up, you want to watch the introductory video. This takes about 10 minutes and give you the basic gist of it. Yeah, yeah, I know, you never watch those things. Yet for every person I’ve introduced to Freebase, after about their 5th question, I find myself saying “look, just go watch the video”. So, go watch the video.

### 2. Search for stuff

What are you into? I’m into tall ships, so the first thing I did on Freebase was search for tall ships. The search box is at the top right of most pages on the site. Go search for something now — anything, really, whether it’s a film or a person or the ebola virus — and you’ll see that there’s a bunch of stuff in Freebase, mostly seeded from Wikipedia.

Click around a little. You’ll see in the search results that some results have “types”. The most basic type is “Topic”. I think of this like LambdaMOO’s `$thing` — it basically tells you nothing, and only has attributes necessary to the system itself, like creation date, name, and so forth. But there are other types, and individual topics can be members of as many of them as are necessary. For instance, Steve Jobs is, among other things, a Computer Designer, a Company Founder, and an Adopted Child — this last one being a user-generated type, not yet promoted into the official hierarchy.

So, if you were simply using Freebase as another kind of Wikipedia, this provides you with a new way of navigating around. In addition to clicking on ordinary topic links, you can also click on any type and see other topics that have that type. For instance, click on “Computer Designer” and you see a list of other computer designers, including Alan Turing. It’s like Wikipedia’s categories, but more rigorous, and with real inheritance.

### 3. Query

So what’s the benefit of all this structure? Well, obviously, that you can query it programmatically.

Asking Wikipedia for a list of computer designers born before 1930, or tall ships greater than 30m in length based in Australia, or film directors who have also acted in films, is nigh impossible. For Freebase — and its underlying technology, Metaweb — it’s easy.

(Just to clarify: the company that built this thing is called Metaweb, as is the underlying database tech; Freebase is one application of Metaweb, but it’s easy to imagine it used, for example, within an enterprise as a knowledge management system.)

So, there’s the Metaweb Query Editor, a web-based interface to MQL (the Metaweb Query Language) or you can use one of the APIs in your language of choice. If your language of choice is Perl, you have a choice between Metaweb or WWW::Metaweb; both do roughly the same thing, but the former is my own module, so I’ll use it in the following examples. There are also APIs for other languages (Ruby, Python, etc) but I’m not really up on them so can’t provide examples.

MQL itself is a JSON-based query-by-example language. There are a number of examples available on the Query Editor page, linked above, but here’s one I made myself:

{
“ships”: {
“query”: [{
"type": "/user/skud/default_domain/tall_ship",
"rig": "barque",
"name": null
}]
}
}

This says, “Give me all topics of type ‘Tall ship’ (one of my own private types) whose rig is ‘barque’ and tell me their names.” Or, in pseudo-SQL, `SELECT type, name, rig from Metaweb WHERE type = ‘tall ship’ and rig = ‘barque’`.

The result is JSON output that looks, in part, like this:

{
“rig”: “Barque”,
“type”: “/user/skud/default_domain/tall_ship”,
“name”: “USCGC Eagle”
},
{
“rig”: “Barque”,
“type”: “/user/skud/default_domain/tall_ship”,
“name”: “Belem”
},
{
“rig”: “Barque”,
“type”: “/user/skud/default_domain/tall_ship”,
“name”: “Pommern”
},

If you have the Metaweb Perl module installed, you can do all this from the command line using the provided `metaweb` script:

metaweb < barques.json > result.json

Or, if you want to embed it in a Perl application:

#!/usr/bin/perl

use strict;
use warnings;
use Metaweb;

my $mw = Metaweb->new();
my $result = $mw->query({
name => ’ships’,
query => [{
type => "/user/skud/default_domain/tall_ship",
rig => 'barque',
name => undef,
}],
});

foreach my $r (@{$result->{content}}) {
print $r->{name}, “\n”;
}

(The Metaweb module lets you query either in raw JSON, or using a Perl data structure with a similar shape to it, as you can see.)

### 4. Write applications

It’s obvious from this that it’s pretty easy to write applications using the Freebase data. Since all of Freebase’s data is licensed under Creative Commons “CC-BY” license, all you need to do is attribute the source of the data.

I’ve mentioned the Perl API; Metaweb have also provided a Javascript one called MJT (pronounced “midget”) which lets you write entire applications in a single HTML file. There are a number of them listed in Freebase itself, but one great example is FMDB, an IMDB-alike written using MJT in a single 24kb HTML file.

Obviously the potential is enormous. I’ve got dozens of projects I’ve considered doing then wimped out of because I’m too lazy to do all the database work; with Freebase, I don’t need to.

### 5. Modify data

You need an invitation to the alpha program in order to login and modify data on Freebase, but once you have one, you can use the Freebase website itself, MQL, or an external application to dink around to your heart’s content. You can create your own types, add attributes, and link topics to other topics in a myriad of ways.

If you’ve got some collection of data which you own the rights to, or can get permission to upload to Freebase, then you can import it wholesale; unlike Wikipedia, Freebase doesn’t have a stick up its arse about “notability”. One friend of mine is planning to upload data about as many blogs as she can find; another is looking to import information about light aircraft; someone at Metaweb was telling me they’re getting hold of a database of generic and brand name drugs to include. The only limits that they say they have is that the data might be of interest to other people.

If you’d like an invitation so you can write/modify Freebase data, comment here (ideally, telling me what you’re planning to do with it) and I’ll send you one.