Rufus Pollock on creating an Open Knowledge Society: transcript

Key points

• Open Knowledge is a non-profit network working since 2004 to open up information and help people use it.
• A priest was wrongly told he needed a heart bypass; the case exposed over a thousand misdiagnosed patients at one hospital, and hospital data showed red flags that could have raised suspicion earlier.
• Open government spending data let the UK government CTO spot departments each buying the same expensive report, because the data was open to be searched.
• Data is still often locked up, and it is also hard to get and use, which Pollock calls the friction problem.
• Containerization cut shipping costs and time dramatically; data packages are proposed as the equivalent: simple, opaque to content, and allowing moderated diversity.
• He warns that the web is becoming centralized and urges open, distributed infrastructure so the knowledge society stays open rather than closed.

Chapters

1. 0:02About Open Knowledge and its work
2. 1:30The heart bypass story at a California hospital
3. 7:13What hospital data could have revealed
4. 9:14UK spending data and everyday questions
5. 10:42Data is locked up and hard to use
6. 12:20The shipping and logistics analogy
7. 15:07Containerization and its cost savings
8. 18:26Data packages as containers for data
9. 22:02Wikidata, open versus closed, and centralization

Full transcript

0:02Thank you. It's a pleasure to be here and to get to talk to you. I'm going to talk about creating an Open Knowledge Society. I'm just going to start at the very beginning and give you a tiny bit of information about myself and the organization. Open Knowledge, we're a non-profit, we're a global network. We've been working since 2004 to open up information around the world and in particular also to see that used and useful to empower citizens, to empower organizations, to empower researchers, to empower entrepreneurs, to answer questions.

0:32that matter and drive positive change and we do two particular activities first of all we open up information we bug governments we support governments we talk to corporations we talk to researchers and other kinds of and ngos to open up essential content and data starting all around the world and we also then turn if you like work on turning data into knowledge and and change and that's obviously a huge thing one where we only have kind of selective and we work a lot with other organizations including in fact Wikipedia Wikimania but many many others from civil society organizations to governments and you know the some of the projects from CECAN to open spending to school of data giving people skills one of the things we focus on today is as all data is starting to be available substantial amounts open data but are there the skills in civil society in government in the media and others to take advantage of this.

1:30So, I'm going to start with a story about medicine, about healthcare, and it starts, actually, it starts in 2004 in Shasta County in Northern California, and a Catholic priest, in fact, a relatively new Catholic priest, he'd previously been an accountant in Las Vegas, so this was quite a career change. He was now a Catholic priest. It's sad to say, actually, for this bad later story of this, he actually got defrocked for sexting when I researched this more recently, but that's a different part of this story. But at this point, he was a Catholic priest named John Karapi, and he experienced some chess planes. And Shasta County is relatively rural.

2:21He actually did a lot of fishing up here and then traveled around. He was quite a successful Catholic priest. He was, you know, he went out on the road giving big talks and stuff. But he got chest pain. And he was in his 50s. And obviously, you know, you might worry about heart, you know, heart disease, heart stuff. So he went to his primary care physician who recommended, he referred him to what was at the time called the Reading Medical Center, run by a for-profit healthcare chain called Tenet Healthcare, and he had a cardiogram done, an angiogram, and this is...

2:56Oh my God, you know, he got called up four hours later after he left Osport, you know, you have to come in for a double heart bypass right now. You're at serious risk. It's a real issue right now. We're going to book you in two days from now for a heart bypass. Now, this is obviously quite a shocking and stressful piece of news. He went home. I mean, he was at home, but he called up his friend, who was still, by the way, this was one of his best friends, was still an accountant in Las Vegas. and his friend said to him well my partner is actually a senior a senior nurse at the local hospital why don't you just let me give her a call and just check about this get a second opinion so he called up his partner and she told him you know the phone got relayed was that basically it was really unlikely it was this urgent you know it wasn't like you know it's not it's not like you know you've got a kind of aneurysm or something you have time if you've got an issue with your heart. You've got a week or two. It's not like, you know, it's not like you have to have it done tomorrow. Why don't you come down to Las Vegas to my hospital and I get a second

3:57opinion? So he set off. He came down to, uh, to, went down to Las Vegas, flew down, had a second opinion. You're fine. There's nothing wrong with your heart. You're absolutely fine. Good condition for your age. So obviously he was now a bit shocked, you know, was he at death's door? Was he actually absolutely fine? And this was just a bit of heartburn after lunch. So they got a third Third opinion, third opinion, you're fine, there's nothing wrong with you. So, they went, they kind of got a bit upset about this. I mean, he'd been very stressed. It's a very stressful experience thinking that you're seriously ill. It's a really, if you're not aware, I mean, I wasn't aware before I read about this, but it's, you know, very serious, obviously, operation, having a heart bypass. They cut you open, they split your ribcage, essentially, if they do this. It's a really serious operation.

4:43so they went in to see the CEO of this hospital this is a private hospital right has a CEO in the United States and this is an issue this is a real concern what's going on the guy was like don't worry mistakes happen people can misread angiograms it does happen that you know and they asked other doctors and it's true you know it's not it's not a perfect science but they persisted a bit on this and they kind of got set and they kind of started asking around and And there was a bit of a rumor mill in this area, in this town in California, that they were a bit trigger-happy on the heart bypasses.

5:20In fact, it was said among some of the, it was a joke among some of the, you know, GPs in the area that you mustn't get a flat tire in front of Reading Medical Center because you'll end up with a heart bypass. If you go in there, you'll come out with a heart bypass. And they kind of got suspicious and they started talking. And in fact, ultimately, they got in touch with someone from the FBI. And the reason for this is the FBI interest in the United States. This hospital will charge the government money. It's a private hospital, but it does Medicaid and Medicare. And if someone is disfrauding the government, the government can get involved. The FBI can get involved. Anyway, for various reasons, an investigation started. And it turned out that more than a thousand people over a period of about a decade had been incorrectly diagnosed with heart problems and given serious treatments.

6:05and often involving a heart bypass or a serious operation. Some of the people had died after complications. Some of the people had been left with serious health problems for life. Some people, for example, their chest. One guy, his chest had never knitted back together properly. His ribs had never reformed, and they constantly moved in his chest, causing serious pain. So more than 1,000 people had been mistreated. And the reason for this, sad to say, was felt to be profit. Performing heart bypasses on healthy people is very profitable in the United States.

6:39You make hundreds of thousands of dollars in medical health insurance payments when you perform a high bypass on someone. This was the most profitable hospital in the tenant healthcare chain in the United States for this reason. The doctors who performed, who were in the cardiographic department, were making million-dollar salaries. They never, sadly, well, they had a civil suit, they never went to jail. Now, what was interesting, why has all this got to do with data?

7:13This hospital had two, there is actually a data set in the United States, it's not an open one at present, but it was, there was one starting to be collected in the 90s, it was only in the 90s, which had information on the performance of hospitals. And this hospital had two red flags if you'd looked at the database. One is that it had an incredible number of heart operations for the number of people it served. It was doing a lot of these kind of operations for the number of people in the catchment area. The second was that it had incredible performance.

7:45It had a really good mortality rate for heart operations. because operating on healthy people is really good for your statistics. Now, taken together, that's really odd, though. If you operate on a lot of people, in general, you should either be reverting to the mean or if you're, in fact, taking at-risk people, often it's the case that people operate a lot. For example, UCL, University College Hospital in London, is one of the best hospitals in the UK. And yet, often, its mortality performance for certain operations is very poor because it takes the hardest cases.

8:19So often, if you were doing, if you were an incredible hospital heart bypass, you'd end up with some of the hardest cases, and you'd actually have a poor mortality record. So together, these two things should have been a red flag. I think this brings me to a point about data and the power of data, which is, and about open data. If someone had been looking, if someone had had these suspicions, Loki had been able to look at this kind of data set and put those kind of things together, it would have helped them to transform what was simply a suspicion, a general suspicion, into an actual kind of hard hunch that something was wrong. And this point, which is if you know your free and open source software, you'll know the first of these citations is famous as, to many eyes all bugs are shallow, to many eyes all anomalies are obvious. We have this immense amount of data but we need people to look at it. And the fact was, often people, even if the data is available inside government, not the right people are looking at it. To To give one other example, if Mike is still here in the audience, I don't know if he is, he'll love.

9:14I was told the other day by the CTO of the UK government that he'd be able to save six million pounds in 15 minutes, which I think is doing even better than Mike is doing, which is pretty impressive. By being able to look at government spending money, UK now publishes open data on spending. And by being able to look through what departments were spending, he realized that the departments were all buying the same £50,000 report, there are £50,000 reports, believe it or not, or something like that, that they would buy 15 copies of it. Each department was buying its own. And just being able to see that. Now, government could have looked at that before, but it was only when that data became open data, and when the government's CTO didn't have to start a digital project, but could just go online, search for two seconds and find something, that you made that saving. And at this point, again, the best thing to do with your data will be thought of by someone else. If that data just sat in the Treasury, would the government CTO have been able to make the saving?

10:06Probably not. It needed someone else, somewhere else to do that. And there's so many of these questions on everyday lives. Is it safe to cycle? I don't know if you know, last autumn there was a whole scare in the UK that there was a whole bunch of cycling deaths. Was cycling getting more dangerous? Was it the weather? Or was it just statistical chance? Where does our money go when we pay taxes. What happens to it? What does it get spent on? Where? And climate change, you know, from the smallest things of the everyday on my journey, my commute to work on my bicycle, to big things like climate change, data is essential to be able to address these problems.

10:42Now, when I come to the problem, I think that we generally, I think, as a community, the open community, but especially open knowledge, are looking to address, which is there are a lot of parts of getting data from using information to drive change in the world. And most fundamentally, I'm not learning on this list, is that someone ultimately has to use it. But we, I think, in general, have to focus on particular challenges. We can't boil the ocean. We have to pick our fights. And for us, I think, to start with is to say, looking at many of these examples, the data is locked up.

11:12the data's not available. It's not available to me or you. It was in most many countries in the world, you sometimes struggle to even get the budget, let alone get it in a form that allows you to look at it, let alone get all of the spending data like you get in the UK. On climate change, up-to-date data on climate change and CO2 is still often kept. You have to pay, by the way, tens of thousands of pounds, the OECD, to get one of their key data sets on climate change. So data is locked up, and data is hard to get and use. Now, in some sense, we've seen incredible progress over the last decade since we were founded, and particularly in the last five years on the first of these, data being locked up.

11:47So I'm not going to go on about it today, actually. I'm going to focus on data is hard to get and use. I'm going to get really specific, because I think this is a very informed audience. I don't want to go on too much about open in general. I'm going to talk about something I call frictionless data. I think part of my point here is that data generating, and in a way we could forget open data for a moment, is too hard to get and use, particularly in a distributed community, particularly in a community that isn't one corporation, that isn't Google, that isn't the NSA. The NSA is probably the biggest data processor in the world.

12:20But if we're a distributed community, if we're a collaborative community, if we're a community that goes across individuals, that goes across organizations, there's too much friction. And I'm going to push an analogy very hard with you today, and I hope you'll bear with me. It's a shipping analogy, and I like to think of it as logistics. Who here, I'm going to do a bit of interaction, who here has gone and bought some food in the last week? Maybe if I really wanted to, I mean, hardcore geeks here, maybe I should ask, who hasn't gone and bought food in the last week?

12:52No one shamefully puts their hand up. No one has gone, I mean, you know, actually gone out to a store and bought it, not, I don't mean consumed it, but gone out. Okay. But do you ever think, I mean, occasionally it hits, you know, you go into a store today and you just think how amazing it would be for a medieval peasant. Just the incredible choice. You go into a supermarket and say, and the incredible choice. And you stand there, just in the fruit section, and you think how far some of this stuff has come. I mean, I'm not trying to wait whether that's good or bad. I'm saying it's incredible, the choice in front of us. And the miracle of logistics that goes into that. Stuff has been shipped sometimes halfway around the world, or it's even been shipped from Scotland or from just outside London. But it's been picked by someone, it's been packed up, it's been put in logistics lorries.

13:37And you know one of the things about medieval times that was incredible to me when I was reading about it recently, is when famines happened, it often wasn't that there wasn't enough, for example, corn grown in the UK. You could actually have localized famines. You could have an issue where there was enough corn in Manchester, but not enough in London, because the cost of transporting corn was so much from place to place. And so the incredible thing for me, I think, about transport is how essential it is in our daily lives. When I go to get those ingredients to make a cake, all of the stuff that's gone from the corn being cut in the field to it being transported to me. Now, the reason I'm having this analogy is it's about data and information. Ultimately, we want to make a cake or we want to eat dinner. We want to produce a beautiful graph or create some new insight or understand whether hospitals are killing people semi-intentionally for money. And that involves getting information together. It's like I go to the information store.

14:32I want to get all of that stuff together, you know, if you like, and I want to bake my cake the cake is the end analysis that I make the insight I have the visualization I make And at the moment, and I'm going to push the shipping analogy, specifically about shipping, we're almost in an age where I have to go out into the fields and almost get the corn myself. Maybe I don't need to grow it, but maybe I need to cut it. I need to mill it myself to make flour, and then I need to cart the flour back, and I need to go and get the eggs from the hens. A lot of the time when you do data projects, you're doing all of that. You're not just focusing at the end part of it.

15:07So I'm going to really push this shipping analogy for a particular, specific point I'm going to make about a particular revolution in shipping. So back in 1955, shipping in particular looked like this. There was a particular part of this, which was loading ships. Loading ships was a big deal, right? It was, in fact, I'll come to it, it was a large part of the cost. And this is what it looked like. people by hand would load ships and it was manual, it was slow and it was costly and it was also dangerous. I don't know who here has seen On the Waterfront?

15:42Anyone? Yeah, some people have seen On the Waterfront with Marlon Brando and On the Waterfront is about, well I guess the politics of Steve Doering in the United States, right? These people, this is a whole union, it was a very powerful union that loaded ships, they were called Longshoremen or Steve Doers and if you remembering on the waterfront, if you've seen it, one of the guys who's protesting against the evil union gets crushed in a ship by people dropping stuff on him. It was a dangerous profession and it was incredibly inefficient. And in 1957, the first containerized ship was built. And it has revolutionized shipping.

16:16There aren't, I mean, whether you think it's good about it, today, there are very few people around. Very few people get injured loading ships. It is automated, it is mechanized. And this is one part of the journey, right? This is one part of the stuff that gets food, that gets your laptops, that gets your phone, from wherever they get made to you when you go in the store. And containerization has revolutionized that. and data is shipping pre-containerization. That's my argument. Today, we load, we do all of this kind of loading in my hand. I just want to give you an idea of how revolutionary this is in terms of cost in shipping. When I say how revolutionary, this is what it used to cost to do shipping.

16:54Okay? So, in 1956, this was the first containerized ship, and you can see it compared to what was called a break bolt, traditional loaded ship. Okay? It's the second line. The loaning cost per ton was $5.86 versus $0.16 for containerized shipping. That's approximately a 50 times improvement in cost. Look at the tons per man-hour, right? The year, this is over time, but 1956, this is an average.

17:25By 1976, most stuff was containerized around the US. In 1959, you did 6.6 tons of loading by hand per man-hour. In 1956, you did 4,234, which is close to being a 6,500 improvement in performance. And hours in port went from 504 to 18. and just to give you a sense of the statistic in terms of shipping goods around it used to be that the five miles either side of port which included the off offloading accounted for over 90% of shipping stuff around even in 1955 it was the off unloading and offloading with by hand they consume 90% of the cost of this stuff and that's what happens for those of you know who do stuff with data today that's what it's often like before you get your beautiful visualization you

18:26spent days if not weeks collecting data together cleaning it up working out what's wrong with it fixing it up getting it from one machine to another and I want to make an argument then for something very specific and I'm gonna say why was containerization so good I'm gonna argue there were three things that were great about containerization Right? Just to be clear about containerization, it was just about putting containers on things. It didn't change the stuff inside. It didn't do anything else. It just put a standardized box around the stuff you were shipping. It was simple. A steel box is very simple. It was opaque. You could ship iPhones. You could be shipping bananas. You could be shipping cars. It's all the same to the container, roughly.

19:12It all goes inside a container somewhere, and then it gets loaded on the ship. And the third thing I would call, which is a bit odd, is moderated diversity. Which is that not all containers are the same size, in fact. They may look like it, but there are different versions of containers, and I'll come to this. There is allowance for diversity, but it's under control. It's not any kind of container, but there's moderated diversity. And so I'm going to introduce the analogy for data here. This is the point I'm making called data package. And you can find out more at data.okfn.org, Open Knowledge Foundation website.

19:46So the data package I'm arguing is the analogy for the shipping container for data. So I'm not going to go, this is a keynote talk, so I'm not going to get into the dirty details of the tech of it. But what I'm saying is it's an incredibly simple wrapper around your data. It's like a shipping container. It's pretty bare and simple, and it says there's this data package.json, and you can put data inside this thing. The second thing about it that I'm going to come to, which we could talk more about, but it's any kind of data. Data packages ship any kind of data, just like containers can. They're opaque to what goes in them. They can ship geospatial data, they can ship tabular data, they can ship Wicked images, they could ship video, they could ship whatever you like. They're focused more on structured data.

20:33and the third thing is what I call moderated diversity so just to go on about this one for those of you who are your shipping geeks I don't know if there any of you here but I only I had to I had to get up to speed for this talk that's the container on the left is a real Rolls-Royce it's a very specialized container it's a pallet wide exactly one standardized pallet wide by the way I didn't know this before but pallets if you know what I mean by a pallet wooden pallet they're standardized there's an ISO standard for pallets about their size. And this is exactly one pallet wide, 45 foot long, high cube shipping container. So it's a shipping container but one that's slightly specialized. And that's the same with data packages. There are data packages just for tabular data that say you've just got to ship this kind of thing. It can't be, it's like a pallet. It can't be bananas. It has to be exactly this thing. It has to be CSV data in fact. It has to be tabular data. It has to to be a CSV and so on.

21:27This is moderated diversity. And so I kind of want to say on this, which is say, this is not, I'm coming in some sense with some humility here. I'm not trying to pitch you some incredible vision. But I think that it's like the shipping container, that it seems rather humble and simple, but it has revolutionized our ability to do logistics. And right now, part of our issue is around data logistics. Our ability to efficiently get data from A to B and to cut out some of the huge amount of effort that is going on at the moment shipping data around between different places.

22:02One can see this in some sense with Wikidata, right? Most of the data going into Wikidata already exists somewhere in some other system. Yet I get the impression, and rightly, that quite significant amounts of effort are being made to transport it across and integrate it in that effort has to go down if our community is to flourish if we all be able to ship data around and therefore build a distributed collaborative community that we want to build so I'm going to conclude and I want to kind of loop back from this very specific example to the big picture which is that we're becoming this knowledge society and in this knowledge society there's always that commonplace that knowledge which is power.

22:51But that is even more true today, and will become more true during the century. And it's not clear, to be honest, what way it goes. Many of us, I think, have a slight inclination to the techno-utopianism, that technology will generally improve things, and digital technology even more so. It's been incredible what it's allowed us to do. But it's not inevitable. There is immense power and wealth to be held by controlling information.

23:21and we're going to have to have a choice and in fact I would imagine some degree of significant fights about open versus closed and I think this vision that we have of the Open Knowledge Society is one that we want to see built around open rather than closed around collaboration rather than control and around sharing rather than exploitation and fundamentally about empowering versus exploiting people and that is something where we have to think hard and it's something where we have to architect systems that allow us So to go back to why I've gone on today about the shipping container, it's fundamentally, I think, at this point, we're starting to win.

23:58I think it's so far to go, but we're starting to win in terms of open data. But one of the challenges we have today is we have very little infrastructure. A lot of our infrastructure at the moment seems to be pushing for largeness and centralization. Does anyone remember the net like 14 years ago when I, I mean, in the 90s? I think a number of websites. I go to fewer websites today than I used to by a long way. The web has become centralized at the present in significant ways. Even Wikipedia is an extraordinary example. It is amazing. But when I first run into web, I would go to many different websites to find something out. Now I maybe just go to one.

24:39and that is a dangerous tendency and it's a point where things like shipping containers actually matter because the infrastructure determines how we can collaborate and our power is in our distribute is in our community is in the wealth and breadth of it but we need the infrastructures that allow us to effectively collaborate at scale and things such as shipping a part of that that efficiency is fundamental to how we can set up an open information and open knowledge ecosystem so we want to make sure that we become an open knowledge society and not a closed one and I want to leave you with that that we empower ourselves to answer the questions to go to the curiosity in us to ensure that that ability to satisfy our curiosity to pursue it remains thank you very much

Source and licence

Recording: File:Keynote Speakers IV - Applications - Creating an Open Knowledge Society.webm by Wikimania 2014 (2014-08-08), via Wikimedia Commons. Licence: CC BY-SA 3.0. The transcript, summary and chapters on this page are shared under the same licence (CC BY-SA 3.0).

The transcript is machine speech-to-text by the TidyTools Audio & video transcriber; the summary, key points and chapters were written by Claude from that transcript. Both may contain errors: check the recording before quoting it.