A couple of interviews

I’ve recently been interviewed by a couple of different blogs, and thought I should link them here:

  • The Ada Initiative blog interviewed me about Growstuff, pair programming, and social justice. They’re having a fundraising campaign to support their work with women in open technology and culture, by the way, and if you care about those things you should definitely donate.
  • Maciej from Pinboard interviewed me for the Pinboard blog, also about Growstuff, which (as you may recall) he funded to the tune of $37 back in January. It’s good to have such support from our investors ;)

Go, read!

Start your commit message with a verb

I’ve been pair programming with a lot of different people, with a variety of skill levels, on Growstuff over the last year. One thing I’ve noticed is that some people freeze up when it comes to writing a commit message. They type “git commit” and then sit there for a minute going “uhhhh”.

I understand this. It’s hard to convert maybe an hour’s hard work in code into a short sentence of English. How do you compress such complex ideas? How do you even make words, when your brain has been deep in code?

So here’s the tip I give to my pairing buddies who freeze up when it comes time to commit, and I offer it here for free: Start your commit message with a verb.

“Added…”
“Fixed…”
“Removed…”
“Refactored…”

The rest usually comes easily. What did you add? What did you fix? What did you refactor? Grammatically, this is the direct object, and starting with a verb works as an effective prompt to figure out what it might be.

Sometimes you need an indirect object as well (“Added planting_count to crops”) or a reason (“Added planting_count to improve performance”) but really, if you can get a verb and a direct object, you’re most of the way there. And it’s certainly better than “WTF!?” or “yay bugfixes!” or “.”, all of which I’ve seen as commit messages.

You’re welcome.

(Of course, if you don’t freeze up when you have to write a commit message, then keep doing what works for you.)

The problem with doing one thing well

You’ve probably heard the tech startup aphorism “do one thing well”, or a variant on it. “Don’t try to do too many things”. “Focus.” Whatever.

I’m not very good at following it, as is pretty apparent from what I’m working on. Growstuff has several things it’s trying to do (crops database, garden journal, seed sharing, community building), all interlinked.

Every so often someone points me at a website that does just one thing of the set of things we’re trying to do. For instance, the other day I got an email from a Transition Town contact, suggesting I look at RipeNearMe, which offers produce sharing/trading. If you’ve got extra lemons or zucchini or eggs, you can offer them for sale to people nearby. Great! The website looks fantastic, and they’re starting to get people listing stuff. (If you want to see some examples of what they’ve got available for trade, check this neighbourhood near me, which has a few things listed, though it’s sometimes slow to load.)

The Transition contact went on to say that maybe Growstuff should “join forces” with RipeNearMe, so as to avoid duplicating effort.

The problem is, we can’t. RipeNearMe doesn’t have any way for us to integrate with their data. There’s no API, and their terms of use are restrictive and prevent us from using their data in any way. (There’s also no open development community we could join, but that’s not what I’m discussing in this post.)

There are a lot of gardening sites out there that do one thing, often very well indeed: a Q&A forum, a seed swap site, a database of planting times, garden layout tools. But when we talked to people who used them, they said “I used this site for a while, and it was useful for that one thing, but I really wanted $other_thing as well.” Usually there is another site that offers the desired feature, but it doesn’t integrate with the first one. As a gardener, you need to use a dozen disparate sites, re-entering your garden data in each one, and having to check in on each of them regularly to keep them updated. It’s no wonder that so many gardening sites, flourishing at first, start to die down after a season. Before long you can see weeds growing everywhere.

That’s not to say that open data and APIs solve everything — I’ve written before about how importing data is hard — but without them it’s impossible to integrate anything.

I’m reminded of Anil Dash talking about the web we lost: heavily interlinked, easily syndicated, less silo-ed. I’m also reminded of the Unix philosophy and especially of pipelines. Unix commands “do one thing well” — sort a series of lines, count words, spit out the contents of a file — but they don’t work alone. You can chain them together to say things like “show me the wordcounts of all these files in descending order”, or express even more complex ideas, as if building a tower from blocks.

Now think of that in terms of gardening websites. How awesome would it be if you could say “take my garden layout from SmartGardener and import it into my to-do list on Growstuff, then cross-reference it with the planting dates on Gardenate and the weather feed from the Bureau of Meteorology, and tell me when to plant things. Then when I harvest the results, let me post my excess across to RipeNearMe and, heck, why not CraigsList too?”

That’s pretty unlikely to happen, but until it does, I feel pretty justified in not doing “just one thing” with Growstuff. “Just one thing” only works if you can integrate with other things. If you build one amazing feature and put a fence around it so nothing can get in or out, what’s the point?

And a third thing:

3) I’ve been sitting on this for a little while, but it’s been announced now, so: I’ll be keynoting Open Source Bridge in Portland, Oregon (USA) in June. I know a bunch of my people will be there and I can’t wait to see you all. If you have never been to Open Source Bridge before, it’s one my my favourite conferences, bridging (get it!) software and social responsibility in a way that you don’t see many other places. I’m pretty sure I’ll be talking about Growstuff and how growing food is like writing software. It is, really!

So hey, two things:

1) Growstuff is live. Go check it out. It’s what we’re calling a “soft launch” and we’re still building features at a cracking rate, but it’s there and it works and we want people to try it out. (What’s Growstuff? Haven’t you been paying attention? It’s a social website for vegie gardeners. It’s an open source project. It’s an app platform AND a dessert topping.)

2) The Disreputable Order of Hopperites, a Melbourne gathering of geeky/technical women, is having its second meeting next Monday. It’s a really chill, fun group, with interesting talks. If you are in Melbourne, identify as a woman/girl/female, and are into technical things, you should come! Register at the link above. We still need another speaker, too, if you have a tech topic you’d like to talk about for ~15 mins.

Why Growstuff is Open Source

This was originally posted on the new Growstuff blog, which I set up the other day. I also set up a fortnightly newsletter, to which you should subscribe if you want to keep up with what’s happening with Growstuff as we count down to our public launch, in (eep!) about 2-and-a-bit months.

My background is in open source software, and I’ve been using and producing it for almost twenty years. Sometimes it’s easy to live in the open source bubble, and fail to notice that there are areas where open source software is not common or standard. Over the past few months, working on Growstuff, I’ve attended a number of events for social enterprises and sustainability, and checked out dozens of websites aimed at food gardeners or people trying to live more sustainable lives. Venturing outside my former bubble, I’ve found that open source software is the exception rather than the rule in these areas, so I thought it would be a good idea to talk about why Growstuff is open source, and why we think it’s important.

It’d be traditional at this point to talk about what open source software is, and to give a quick definition. But open source is at least three things, and each needs its own explanation.

First of all, open source is a political movement that aims to change the power balance between software creators and software users. When you use traditional software, you have to take it as-is. If you don’t like it, you have few options. Software makers can change the software any way they like, charge you what they want for it, or withdraw their support for it at any time. You’re locked in an unequal relationship with them, where they hold all the power. Open source software gives power back to the users, letting them — us — understand how it works, use the software how we want, modify it if we need to, and access it regardless of who we are, where we’re from, or how rich we happen to be.

It does this through special software licenses. You’ve probably clicked “Accept” on a lot of software licenses in your time, and open source licenses are just like this, except that they offer you (as a software user) a bunch of rights, where other licenses typically take them away. An open source license says that you have the right to use the software for any purpose whatsoever. It says that you’re allowed to read the source code — the underlying program that makes the software run — and to change it if you need to, to suit your needs. It says that you can share the software freely, passing it on to friends or colleagues without having to pay license fees or worry that the software creator will come after you. In some cases (as in the license Growstuff uses) it says that if you modify the software and share it with others, you must use the same open source license, to make sure that people down the line have the same rights you do, and to share the love as widely as possible.

Finally, by changing the balance of power between software creators and users, and enshrining that greater equality in a formal document, we open ourselves up to a more collaborative way of working. Software creators and users are able to come together to build the software they need, and users can even contribute directly to the software itself, by modifying the source code and offering their changes back to the original creator. Over the years, open source software developers have learned all kinds of effective ways to work together as distributed, often international teams, and to engage their user communities in developing something that they really want to use and in which they feel a sense of ownership.

So what’s this got to do with social enterprise, sustainability, and Growstuff? In my mind, open source, sustainability, and social enterprise are closely intertwined, to the point where I feel that choosing open source is a vital part of the whole picture.

When we talk about social enterprises — businesses that hope to achieve a social good through their business activities — we seldom look at their software practices. But the choice of software to use, or decision to develop software under a closed or open model, has a social impact, just as do the choice of environmentally friendly materials for physical manufacturing, or the decision to employ people from disadvantaged backgrounds. We expect social enterprises to follow ethical business practices; why not expect them to follow software practices that support equal access, transparency, and accountability?

When it comes to sustainability, it’s about more than changing your light bulbs or using a fancy water bottle. Sustainability’s about developing communities and ways of living and working that can survive and thrive in the long term. Open source is a sustainable way of building software. If a company that writes closed software goes under, the software dies with it, but an open source software project can live long beyond the people or institutions that started it. Since there’s a broad community of people familiar with the software, who know how to read and modify its source code, new developers can step up. An open source project is one that builds community and resilience against all kinds of change: exactly what sustainability is about!

These are the reasons why we think it’s important that Growstuff be open source. We want to work openly and ethically, in collaboration with our members, building a community that feels a sense of ownership and deep involvement in the software that runs our website. We want other projects, especially those working in similar areas, to be able to look at what we’re doing and learn from us, through reading or re-using our source code. We want to know that if something happens to Growstuff itself, a new Growstuff — or a hundred new Growstuffs — could sprout up, and that people could continue to benefit from what we’ve built far into the future.

Hurrah, I’m $37 richer!

Just a quick post to note that Growstuff (my open source project for food gardeners) was selected as one of the winners of Pinboard’s satirical startup incubator program. I get $37 in funding, woohoo!

While the $37 won’t pay for much of anything — that’s the point, after all — I’m looking forward to Maciej’s advice and help with getting our name out there, and to getting to know the other winners. I’m pleased to see another food startup on the list (home baked goods via the Internet!), would love to be able to use the pre-hardened machine images for AWS, and can’t help but be excited that a sailing-related startup is amongst the winners. While I don’t play board games much, nor have a kid in school, both those projects sound useful and likely to succeed, too. Congrats to my co-selectees!

Global Shifts conference

Tomorrow I’m off to Global Shifts, a three day social enterprise conference being hosted at RMIT. I’m very glad someone happened to mention it to me last week, just in time for me to register.

I’ve started describing Growstuff, in appropriate circles, as a social enterprise. Lots of people don’t know what the term means, so I’ll just quickly define it: a social enterprise is a business which hopes to achieve a social good, but does so through its business practices rather than the fundraising/donations model that most charities use.

I consider Growstuff to be a social enterprise on several levels. The first is that by helping people grow their own food, we are addressing food insecurity and promoting environmental sustainability. The second is that by aggregating data about people’s food growing activities and releasing it under a Creative Commons license, along with our open source code, we’re freely providing technology to help other people build tools and services for food growers, or to help researchers understand how people are growing food. The third way that Growstuff works as a social enterprise is through our community and development processes: as a non-traditional software project, we offer training/mentoring and a supportive environment for people from non-traditional technology backgrounds or who are marginalised in the technology industry to learn, improve their skills, and take leadership roles.

I’ve been to an uncountable number of tech conferences over the past decade or so, but Global Shifts will be my first social enterprise one. I keep remembering something someone said in an intro session the one time I attended SXSW: “Don’t attend sessions about things you already know. You’ll only sit there being annoyed they’re not covering your favourite topics, and thinking you could do better. Instead, go to sessions about things you know nothing about.” Some of the best conferences I’ve been to have been the ones where I’m stepping outside my usual field — I’m thinking especially about the museum/library/archive events and digital humanities “THATCamps” I attended in 2010-2011 — and I’m hoping that Global Shifts is going to have the same effect: lots of new subjects to fill my brain, and very few where I doze off because I’ve heard it all before.

Here are some of the sessions I’m hoping to attend:

Wednesday

  • Developing your social enterprise idea to a workable model — 2 hour workshop, hoping it will be very useful, but slightly worried that we’ve already advanced beyond it. On the other hand, I suspect the other main contender in this timeslot, “B Corporations (what, why, how?)” will be covering material I definitely know from having pored over their website over the last week or two.
  • Structuring a social enterprise – what are the options? — 2 hour workshop, and I’m very interested in this, because this is one of the next steps for Growstuff.

Thursday

  • Measuring Impact — not the obvious choice (that would be “Paddock to plate, food is leading the revolution”) but another Growstuff person will most likely be attending that one, so we’ll each take notes and swap info afterwards.
  • Making Change: If it’s so good, why aren’t more people doing it? — another one that’s on against a green/environmental session, but I’m planning the same note-swapping deal in this session too. I actually think this session might be useful for insight into why people might not want to use our stuff.
  • SOCIAL DRAGON’S DEN: Real projects pitch to real investors — I want to watch other social enterprises pitching their ideas, and see how they’re doing it — and how we can do it. Not that we’re pitching to investors, but it’s good to be able to explain our project in a compelling way to all sorts of people.

Friday

  • State of Australia — I’ll just quote the description and you’ll see why this is so interesting to me: “What structural supports are in place to get social enterprises up and running successfully? Who and what is available to back-up your start-up? Where do you go when you need support, guidance, direction or simple good advice?” Yes please.
  • Structuring success — this is a session about co-ops, B-Corporations, and other alternative governance models. I actually think this might be more interesting than the B-Corp workshop on Wednesday, since it presents a wider range of options.
  • Leadership and people: how to create a movement — not super psyched by this (suspect it might be a bit Social Media 101) but it looks way more interesting than the alternative (“Can big business save the world?”) so I’ll probably go to it anyway.

So, that’s my plan for Global Shifts. I’ll probably be tweeting from there (hashtag #globalshifts). If anyone reading this is attending and wants to meet up, drop me a line.

Importing data is hard: a rant about integrating open data projects

A few times on the Growstuff mailing list or IRC channel, someone’s excitedly suggested that we should import data from another CC-licensed data set. Each time, I say, “Trust me, that’s pretty complicated,” but I’ve never actually sat down and explained the full gory details of why.

The following is something I wrote up for our wiki so that I could point people at it next time the subject comes up. I thought it might be interesting to a wider audience, too, so that’s why I’m posting it here.

Importing data is hard.

This is a bit of a rant by Skud, who used to work on Freebase, a large open-licensed data repository which imported data in bulk from a range of sources, including Wikipedia, Netflix, the Open Library Project, and many more. She’s had a lot of experience in this area, and learnt a lot about the weird complications of mass data imports.

The simple case

They have a database. You have a database. Your fields are the same. Their API is easy to use and their license is compatible.

  1. map their fields to ours, eg. their “name” is our “system_name”
  2. import data
  3. PROFIT!

Their fields aren’t the same

What if the fields aren’t quite equivalent? For instance, let’s say they have measurements in imperial and we use metric. We’ll need to have ways to convert them. That’s actually a really simple example. Import incompatibilities are more often at a semantic/ontological level. Growstuff has the idea of “crops” and “varieties” but what if the other database only has “plants” with no distinction? Or what if they have crops and varieties but draw the line somewhere slightly different to where we do? These sorts of incompatibilities are more common than not, and massively complicate any import effort.

Some of their data is bogus

Nothing against that other database — some of everyone’s data is bogus! But we need to check it. What “bogus” means will vary from place to place, but it might be spam entries, duplicate records, simple errors, or it might be cruft from their own broken imports. We need to look carefully at every import and make sure we’re skipping as much of this as possible. And this is largely a manual process, since what the bogosity will never be the same twice. You can do this by sampling, of course, but you still need to look at something on the order of a hundreds of records, and know what you’re looking for. Could you spot a mixed-up scientific name on a randomly chosen herb? I couldn’t.

The stub problem

Let’s say we want to import from a database of plant life that lists 10,000 edible plants and their nutritional content. Growstuff has 300 crops at present. We import everything! Now we have 9,700 pages with nothing but nutritional data. Nobody on Growstuff is using them, they have no pictures, they have no planting data, they have no discussions (except maybe spam comments that nobody cleans up because nobody notices). Our “newest crops” page, usually a source of interest, is now just a wasteland of grey placeholder images.

Should we have imported all 10,000 plants, or just the nutritional data of the 300 we already have? Or something in between? The answer is usually “something in between” — you might want that data if and only if you can get other partial data from other imports to make it more interesting.

The best way to do this is to import the 300 and make a note of the 9700. Then later, you can cross-correlate the notes you’ve made from various data imports and re-import those that have, say, at least 3 useful data sources and a picture. But that’s pretty complicated. (Also, see the discussion of repeated imports, below.)

Don’t forget the license

Let’s assume that their data is licensed compatibly — that means CC-BY-SA or CC-BY in our case, since we’re CC-BY-SA and none of the other clauses (ND, NC) are compatible with us. (Ignoring CC-0 and public domain stuff for now — those don’t need attribution at all.)

So by importing, we have to credit them. Now we need some way to represent that in the database. If we do this at the object level, it’s fairly simple: each thing in the database (crop, etc) has many licensors, each of which includes a name for the work (eg. “Katie’s Plants”), a license (eg. CC-BY), a licensor name (eg. Katie Smith), and a URL to link to the original data.

Now we have to display them on the page. Where? Probably at the bottom somewhere: “Some information on this page came from: Katie’s Plants (that would be a link) — CC-BY Katie Smith; SuperPlantDB under CC-BY-SA SuperPlants Inc; etc.”

The license chaining problem

Now imagine that the data on those sites came from other sites. For instance, let’s say Katie’s Plants previously did an import from Freebase.com, and SuperPlantDB did one from Wikipedia. We not only need to credit Katie’s Plants and SuperPlantDB but also those places.

Some questions to consider:

  • Are those second-degree sources, their licenses, licensors, etc available via the API? When Freebase imported images from Wikimedia Commons, we encountered this problem, because the license metadata had to be scraped from inconsistently-formatted HTML. Getting this wrong leads to complaints from licensors whose licenses we’re violating.
  • Do we know what part of the data on Katie’s Plants was sourced from Freebase? Maybe it was the international names, but we’re importing medicinal uses and not touching that part of her data. Does Katie’s license notice express this? Probably not — there’s no requirement in the CC licenses for the attribution to be at the field level, and our own attribution notices definitely don’t operate at this level of detail. Because we don’t have the detail, this means we end up with attribution inflation: pretty soon, every page on Growstuff has a hundred attributions at the bottom of every page.

Sure, we could just choose not to chain licenses, or to do it in some restricted way… but the moral high road here is to respect everyone’s license and attribution, and besides, if you only attribute some contributors, where do you decide to draw the line?

The infectious NC clause problem

This is a subset of license chaining problems. Let’s say Growstuff (a commercial entity using CC-BY-SA) imports from Katie’s Plants (a non-profit entity using CC-BY-SA) which imports in turn imports from Hippie Herbs (a non-profit entity using CC-BY-NC — note the “non-commercial” clause).

Katie’s fine — she imports from Hippie Herbs’ data with impunity because she’s non-profit. She attributes them on her site, and Hippie Herbs is happy. She doesn’t have to use the same license as them because they don’t have a “SA” (Share Alike) clause.

Now Growstuff comes along and wants to import data from Katie’s Plants. Katie’s Plants is CC-BY which is compatible with Growstuff… but what about the data that originally came from Hippie Herbs? We’re commercial, so we’re not meant to use it.

But how do we tell what’s what? Katie probably doesn’t attribute HH at the level of individual bits of data, so we can’t extract just the ok-for-commercial-use bits.

Basically, if you believe in license chaining (and as I said, it’s definitely the moral high road to take, so I think we should) then you have to be constantly vigilant for the taint of NC-licensed data anywhere in the sprawling tree of ancestors to your data.

What if we already have some data? (the merge case)

The simple case is fine for a green-field import with no existing data, which is described above. But let’s say we’re importing data into an area where we already have some contributions from Growstuff members.

  1. map the fields as before
  2. for each piece of data imported, compare with Growstuff
    1. is Growstuff’s field empty? IMPORT!
    2. are the two the same? no-op!
    3. do they differ?
  3. If they differ, do we trust our own community or the import source? Or do we need to adjudicate?

Let’s say we decide to adjudicate. We now need to build an app to let people vote on which one is “correct” — probably best of three or something like that. Freebase did this (multiple times) and I was involved in some of it. We called them “data games” and had leaderboards for who’d voted the most. We couldn’t get enough throughput, though, and sometimes by the time something had been adjudicated, another community member had edited the field on our site, thus invalidating the whole thing. We ended up paying people in developing countries to churn through these votes for us (we used ODesk, but you could use Amazon’s Mechanical Turk or whatever). However, they needed training, and weren’t cheap — even after all the work of setting up the voting queue, there was still considerable expense.

Do we let people edit the data after import?

This came up quite often with Freebase because sometimes they would import from “authoritative sources” who licensed their work specially to Freebase but didn’t generally have a CC license or an open community editing process. For instance, the time when I was talking to some people from the BBC, and one (an older dude) said, “If we gave you our programme data, we wouldn’t want anyone to edit it because we are the experts on our programmes.” This was pretty silly of course — another, younger BBC dude immediately turned to him and said “Ha ha ha, I’ve got two words for you: Doctor Who.” — but sadly these situations are common when you’re dealing with closed/non-community-based/”authoritative” data sources who don’t understand the power of crowdsourcing information.

But even when dealing with compatibly CC-licensed sources with open developer communities, there can still be some problems around the “authority” of the data and how it’s attributed.

Take the case where Katie’s Plants community have spent heaps of time editing their data and are very proud of it. We import it to Growstuff, then our community looks at it and decides that bits of it are wrong and change it.

Do we leave the license link to Katie’s Plants intact? Most likely yes, because our data has theirs in its DNA, so to speak. But what if we essentially deleted all the data from there? This might happen if, for example, we’d imported a picture from Wikimedia Commons then found that the picture was incorrect or inappropriate, so we blew it away. Now we should probably remove the license note. But how do you tell when data has been completely removed as opposed to modified or built upon?

In the Katie’s Plants example, what if Katie’s high quality medicinal plant information gets mixed up with ($DEITY forbid!) low-quality data from less experienced Growstuff members or from yet another import? What implications does this have for Katie’s site and their reputation? Under the license we’re allowed to mess it up because there is no “No Derivatives” (ND) clause, but socially/culturally they’ll be pretty unhappy if we do, and we can expect some backlash.

Repeat imports

Great news! Katie got a government grant and some fantastic press coverage, and her database has expanded enormously. We want to re-run the import. But now consider this case:

  • Katie’s plants, original: “Tomato – red”
  • Growstuff, original: “Tomato – red, yellow, green, black”

When we first imported, we put it to adjudication and found that Growstuff’s data was better, so we went with that.

Now we re-import, and Katie’s data has changed:

  • Katie’s plants, changed: “Tomato – red, yellow, green, striped”

So of course we put it through adjudication again. The correct answer is probably a union of the two sets.

Now, Katie’s database is growing fast, and so is Growstuff. We want to do a regular import from there — perhaps monthly. But somehow along the way, we’ve ended up with different ideas of tomato colour. Every month, their data is different to ours, and we have to keep re-adjudicating the same question: what colour/s are tomatoes? Boring. Our community is tired of playing the voting game, and/or it’s costing us money with our Mechanical Turk people.

So we decide to implement a check: if nothing’s changed on either side since the last adjudication, leave it. But now we have to implement change tracking, not just on Growstuff, but on Katie’s Plants as well. We need to keep a history of changes for every site we import. This is in addition to the infrastructure we’ve had to build to automatically run imports at regular time intervals.

How do we make our data available in return?

Obviously we have an API for people to access our data under CC-BY-SA. But keep in mind the license-chaining effect: if anyone uses data from Growstuff, they will also be constrained by the licenses of all the data sources we import. We will need to make that license information available in the API alongside our data, and make sure all our API docs and related materials explain the necessity of license chaining.

Take a look at Freebase’s Attribution Policy. They use CC-BY, but because of attribution chaining, they can’t just say that — they need a whole page with a wizard to help people figure out how to attribute something on the site. It’s incomplete, too: Freebase decided that they would only require license chaining for “content” as opposed to “facts” (a complicated issue in itself) which means images Wikipedia-based descriptions. They don’t require chained license information for other data sources. This is dubious in terms of the legality and culture of how Creative Commons works — there’s no really firm guidelines on this, but in my opinion the most moral/ethical stance is to always chain your attributions, and Freebase has chosen otherwise. In the past, this has caused some concern from the owners of other data sources that were imported to Freebase. Even Wikipedians have complained that Freebase doesn’t enforce their Wikipedia attributions strongly enough. This sort of thing can lead to reputation problems, if not legal ones.

Just the facts, ma’am

One final complication. Various courts have ruled that “facts” aren’t copyrightable. For instance, the fact that the crop “Corn” has the scientific name “Zea mays” can’t be copyrighted. Even if you have thousands of these facts all together, they can’t be copyrighted, because they’re not a “creative work”. They’re just a statement of fact.

This actually throws the whole idea of CC-licensing collections of data into doubt. And yet we have nothing better, so we do it anyway.

Some data projects have come up with various justifications for this. For instance, Freebase says that the arrangement of the facts is a creative work — that what’s CC-licensed is their schema. That’s pretty creative in itself! The thing is, none of this has really been tested. And so most open data projects have some kind of Terms of Service which explains what they think the CC-license is for and how it’s meant to be used. These generally say, “By accessing our data via our website or API, you agree to behave as if this CC license applied to it (even if there’s not a very strong legal basis for that outside this TOS).”

The original idea of CC licenses was to stop people having to write their own terms and conditions of use for their work, and standardise in such a way that people could easily re-use creative content. Yet for data projects, we end up having to make up our own TOS just to apply a CC license, and we’re back where we started — having to peer at a bunch of legalese and figure out what the hell it means.

Of course once you get into the complexities of license chaining described above, you now also have TOS chaining — if Growstuff uses Katie’s data under their TOS, and Katie uses Hippie Herbs’ under their TOS, is Growstuff now subject to Hippie Herbs’ TOS? No idea! I am not a lawyer! I don’t want to be one! I just want to make a website about growing food!

Conclusion

Importing data is hard! That doesn’t mean we shouldn’t do it, but we should go into it with an awareness of the potential potholes, and carefully weigh up whether importing something is the best choice for us at any given time.

Final note

Katies Plants, Hippie Herbs, and SuperPlantDB are all made-up examples. Any resemblance to actual open data projects is coincidental. Freebase, Wikimedia Commons, and the BBC are real, though.

The joys of jobseeking

Technically, for most the last year or so since leaving Google, I’ve been unemployed. I didn’t receive unemployment benefits, though, because I didn’t really need it and because the paperwork overhead seemed higher than I was prepared to deal with. (Plus of course the periods when I was studying or overseas.) But now I’m working on Growstuff and I’d like to get onto the New Enterprise Incentive Scheme, which offers small business training and mentoring and some small amount of funding for a year while you work on your new thing. Thing is, you need to be on unemployment benefits to qualify, and so I recently applied for Newstart.

As a Newstart recipient, I’m required to search for jobs, even though my goal is to run my own business. Whatever, I can play the game. I applied for a number of jobs online, figuring I’m probably over-qualified for most of them, but it fulfils the requirements. Today I got an email back from one of them, asking me to fill in an online questionaire. Obviously, to show good faith in my “job search” I need to do this, but I have to admit that it sapped my will to live.

The first part of the process was an 80 question “IQ” test which included the following questions:

The idea that the Earth is the centre of the universe is

a) improbable
b) intelligent
c) subversive
d) insular
e) astronomical

Such things as language, clothing, customs, color, idicate[sic]:

a) temperament
b) race
c) birthplace
d) location
e) personality

Which of the following is a trait of personality?

a) affluence
b) reputation
c) position
d) withdrawn
e) power

The second part of the questionaire, about the “product” of my most recent work (i.e. my year at Google) was even worse. Luckily their web app crashed and I couldn’t actually complete it.

My prospects as a trainee fleet co-ordinator seem less than stellar.

No, I still don’t want to work for Google.

You think those Google recruiters would know not to contact me, but the other day I got another perky “Opportunities at Google” email from one of them, telling me that they’d found my “online profile” and that based on my experience they think I “could be a great addition to our team!”

Riiiiight.

Since I just deleted my LinkedIn profile, I emailed them asking where they’d found this “online profile”, since it was obviously outdated. Oddly enough, it seems they’d found a page about me on the Geek Feminism Wiki, and were using the rather sketchy outline of my open source background there as justification for trying to recruit me.

The recruiter admitted that the page was out of date, and asked me to let them know what I’d been up to lately so they could add it to their records. Below is a copy of what I sent them. I’m posting it here, lightly edited, for anyone who’s interested, and in the hopes that the next Google recruiter (I have no doubt that there’ll be one) might use that web search thingamajig to find out whether I’m a suitable candidate before emailing me.


Here’s what I’ve been up to for the last couple of years, since you asked.

In July 2010 the startup I was working for, Metaweb, was acquired by Google. I was brought in on a 1-year fixed term employment contract, since the group we were acquired into (Search) didn’t really know what to do with a technical community manager. I attempted to transfer my role over to Developer Relations, but was told that I “wasn’t technical enough” for the job I’d been doing for 3+ years, presumably because I didn’t have a computer science degree and believed that supporting our developer community was more important than being able to pass arbitrary technical quizzes.

Around the same time, Google started to develop Google+. As a queer/genderqueer woman, victim of abuse, and someone who was (at that very time) experiencing online harassment and bullying, I was very vocal within Google for the need for Google+ to support pseudonymity. Google decided not to do that, and instead told people they should use “the name they are known by” while in actual fact requiring their full legal names, in many cases requiring people to provide copies of their government ID when challenged. (Extensive documentation about this is available on the Geek Feminism wiki, if you’d like to read it. See Who is harmed by a “Real Names” policy? for starters.)

When I walked out the door of Google’s San Francisco office on July 15th, 2011, I was very glad to have left a company I thought was doing evil towards any number of marginalised and at-risk people. My first tweet on leaving was to criticise them for it.

Less than a week later I got my first email from a Google recruiter — not first ever, of course; I’d been spammed with them for years, but first since I quit working for them. Here’s the blog post I wrote about it. In case you can’t be bothered clicking through and reading it, here’s the money shot:

If you are a Google recruiter, and you want me to interview for SWE or SRE or any role that has an algorithm pop quiz as part of the interview, if you want me to apply for something without knowing what team I’ll be working on and whether it meshes with my values and goals and interests, if you want me to go through your quite frankly humiliating interview process just to be told that my skills and qualifications — which you could have found perfectly easily if you’d bothered to actually look before spamming me — aren’t suitable for any of the roles you have available, then just DON’T.

The very day after I blogged about that, my Google+ account was suspended, for using the name I was almost universally known by. Over the next couple of months, I campaigned tirelessly for Google+ to change its policies, working with the EFF and other advocates. My work was covered in Wired, The Atlantic, and a number of other mainstream press outlets. Obviously this was to no avail as Eric Schmidt (at the time, CEO of Google) described pseudonymous users like me as “a dog or a fake person” and no substantive change has ever been made to allow pseudonymous use of the service, despite promises to do so.

I returned to Australia and went back to school. I did a semester of Sound Production at TAFE, but it turned out that the sound engineering course I was enrolled in wasn’t really my cup of tea, just like I’d previously decided, back in the ’90s, that university wasn’t for me. Like so many others, I quit my computing degree because I was more interested in the Internet and open source software than in fixing COBOL applications for banks who were worried about Y2K. But then, I’m sure Google’s HR system already knows all about that — if I’d had a degree, you might have considered me worth keeping on last year. Instead, Google’s reliance on higher education credentials causes it to weed out people like me, even though I have a track record a mile long and buckets of evidence to show that I’m good at what I do.

In the end, I’ve spent most of the last year lying in hammocks reading books, working in my garden, going to gigs, hanging around recording studios, doing the odd bit of freelancing, and, over the last few months, travelling around Europe. It’s given me a good opportunity to reflect on my previous work.

Since I’ve been out of the Silicon-Valley-centred tech industry, I’ve become increasingly convinced that it’s morally bankrupt and essentially toxic to our society. Companies like Google and Facebook — in common with most public companies — have interests that are frequently in conflict with the wellbeing of — I was going to say their customers or their users, but I’ll say “people” in general, since it’s wider than that. People who use their systems directly, people who don’t — we’re all affected by it, and although some of the outcomes are positive a disturbingly high number of them are negative: the erosion of privacy, of consumer rights, of the public domain and fair use, of meaningful connections between people and a sense of true community, of beauty and care taken in craftsmanship, of our very physical wellbeing. No amount of employee benefits or underfunded Google.org projects can counteract that.

Over time, I’ve come to consider that this situation is irremediable, given our current capitalist system and all its inequalities. To fix it, we’re going to need to work on social justice and rethinking how we live and work and relate to each other. Geek toys like self-driving cars and augmented reality sunglasses won’t fix it. Social networks designed to identify you to corporations so they can sell you more stuff won’t fix it. Better ad targetting or content matching algorithms definitely won’t fix it. Nothing Google is doing will fix it, and in fact unless Google does a sharp about-turn, they’ll only worsen the inequality and injustice there is in the world.

I guess you’ll want to know what I’m working on at the moment. My current project is an open source, open data project called Growstuff, which helps food gardeners track and share information about what they’re growing and harvesting. It is built on principles of sustainability, including a commitment to a diverse and harassment-free community, to actively supporting developers rather than excluding them based on misguided ideas of meritocracy, and to funding the project through means that will never put the people running the website in opposition to our customers. That means no ads, in case you’re wondering. We’d rather our members paid us directly; that way, we’ll never forget who we’re meant to be serving. I’m working on Growstuff from home, where I can be myself and feel safe and comfortable. I work with volunteers from all round the world, and get to teach programming and web development and system administration and project management and sustainability to all kinds of people, especially those who’ve previously been excluded from or marginalised in their technical education or careers. We get to work on things we know are wanted and appreciated, and we don’t have to screw anyone around to do it.

Let me know when Google has changed enough to offer me something more appealing than that. If you don’t think that’s likely to happen, then please put me on whatever “Do Not Contact” blacklist you might have handy. I know you must have some such list; I only wish you regularly referred to it instead of spamming people who not only don’t want to work for you, but have nightmares about it.

On the home stretch

It’s almost a month to the day since I posted my last travel update. Since then I’ve been to Paris, then to Calais and across the channel to Dover, along the south coast to Brighton and Portsmouth, up to London, stayed a week and a half, then north to York, brief visits to Durham and Newcastle, a few days in Edinburgh, then over to Liverpool, Manchester, Bristol, and five days in Cornwall.

Tonight I was meant to have taken the ferry from Plymouth to Roscoff in Bretagne, then spent a week meandering back through France and across the Pyrenees to Spain and fly home from Madrid. Problem: ferry strikes mean that my ferry’s not going anywhere. Rather than try and arrange my travel to figure that out, I decided I was a bit over high-energy travel and not all that enthused about figuring out a new route through France and Spain on short notice. So here I am back in London, staying at a friend’s place again, for a week til I hop on an EasyJet flight to Madrid and then home.

I suspect I’ll be taking it pretty easy in London, mostly just kicking around and working on Growstuff. I do want to make it to the one museum that was closed during the Olympics/Paralympics when I was last here, but that’s about it. Okay, and maybe another visit to the V&A. Um. Well, let’s just say that I don’t have any particular plans, and don’t intend to work too hard at it.

That said, if anyone wants to catch up for drinks/meals/pair programming this week, let me know.