A couple years ago at TED, Tim Berners-Lee led the audience in a chant of "More raw data now!" so he's out there trying to create the data Web. And your project in Wolfram Alpha is not to deliver raw data but to deliver the interface that allows for raw data to turn into meaning.
Yes, what we're trying to do—as far as I'm concerned, the thing that's great about all this data is that it's possible to answer lots of questions about the world and to make predictions about things that might happen in the world, and so on. But possible how? Possible for an expert to go in and say, "Well, gosh, we have this data, we know what the weather was on such-and-such a date, we know what the economic conditions at such-and-such place were, so now we can go and figure something out from that." But the thing that I'm interested in is, Can one just walk up to a computer and basically be able to say, "OK, answer me this question."
So it's not necessary to, when you have a question, then have to devise the extraordinarily complicated means by which the raw data, you know—finding the data and then crunching the data. It's crunched automatically and magically.
Right. Say, for example, you're trying to find where's the moon going to be right now from where I am on planet Earth. Now, there's underlying data. It's the orbital elements for the lunar orbit, it's the lat/long, it's all sorts of time-zone data, things like that. But there's a certain distance to applying those orbital elements and figuring out, OK, with these coordinate systems we can compute this thing, and so on. That's one type of thing. Another type of thing, it might be, well, you know the value of something and you know some economic indicator and you know the population of some country, well then, you can derive the per-capita thing and you can compare that with every other country of the world. In the second case, it's almost a shallower, almost piece of arithmetic, and in the first case it involves some physics to actually go in and compute from the raw data and answer the question you want to answer. It'd be one thing to say to people, Here's a table of orbital elements of artificial satellites. OK, that's nice, if you happen to have taken some courses in celestial mechanics, then you can go from that to actually figure out where's the International Space Station right now. That's sort of an extra computational step beyond the data. So data is the underlying layer, as far as I'm concerned, from which we can know things about the world. Really, two big things are needed; there's data and there's algorithms. So in the example of things about artificial satellites, the data is: What are the elements of the orbital satellite? And then there's an algorithm, which is: How does the satellite actually propagate in the gravitational field and get me an actual answer to the question I might ask about "When will a satellite next rise over the horizon for me"?
For us, there's two other issues in what we've tried to understand about how one uses data. One is, so this data is out there, but how do I actually access, how do I ask for a piece of data? The next issue is, there might be data on the size of some particular city, but what city am I talking about? Because there might be a slang name for the city, there might be a standard official government name for the city, and so on. How do I actually refer to things? And that's not something that's part of the raw data. It's not something a sensor is going to pick up for you. It's this weird thing that connects the data to our human experience and our way of referring to things.
The machine intelligence, the intermediary between us and our data, needs to understand our language in order to help us access the data.
It needs to learn from us. The data on its own, the sensors could've picked up everything they could've picked up, but it's not something we can connect to unless our systems can understand what we're trying to ask for. And the other point about all this data is, often you can generate lots of results, lots of answers, and then the question is, how do you present these things? And that's another thing that's very human-centric. One could say, Well, here's a result; it's this giant network with a million nodes in it all in this micro-spaghetti, but no human will be able to assimilate that in any meaningful way. One has to feed this knowledge to humans in a way that they can assimilate. And that requires a certain understanding of the human way of absorbing information. That's another thing that's a meta-layer on top of raw data is, so how should you present it to humans? In Wolfram Alpha, one of the things we found interesting is, we've done experiments where we say, let's just give people the answer—one line, the answer. People ask a question like "What's the GDP of Italy?" There's an answer. It's such-and-such a number of trillion dollars. Well, it turns out people actually don't just want the answer. They're much more interested in the plot—what's the history of that number, what is the ranking of that number among countries in the world, what's the conversion of that number to local currency? And people quickly assimilate the ambient context in something that they need, in order to absorb the information.
But how do you define the limits of the ambient context? You could keep going in that direction so broadly that it becomes impossible to plow our way through those thickets all over again.
Right. In Wolfram Alpha, for example, we've gone to lots of effort to try and say, well, what other things do you present first? Then you press the "more" button and you see more, and you press it again and again and again, and you follow the links, you keep going. So it's what's the most important thing, what's the next-most-important thing. Those are all the sort of meta-information that's part of this delivery of computational knowledge that is beyond just the underlying raw data.
Are those sorts of decisions about how to present the information—what the focal length should be—are those decisions that need to be made by humans, or can they be automated to a certain extent?
We've managed to automate them a bit. There are certain criteria, like for example, when you make a graph. We have tried to abstract from experience with humans, what are the design principles for making a good-looking, easy-to-understand plot? And then we can apply those design principles automatically. For example, when you lay out a network, showing this is connected to that is connected to that, we've pretty much completely automated the process of laying those things out, which is something which—I remember years back, I used to get humans to try and do those layouts for me, and it's rather difficult to do a good layout if you know this is connected to that is connected to that. I actually remember doing a bunch of big ones myself at one point where you literally get pieces of string and you're laying things out and putting them in different places, and sometimes it looks good and you can understand it and sometimes it doesn't. That's an example of something where we've completely nailed that one. We completely automate that process.
What about the process of figuring out what makes a good graph in the first place?
Well, one could divine principles that say what will make this good, what does not make this good. One could go and connect those things to some theory of human cognition if one wanted to. What I've ever done has been typically more pragmatic than that. Taking a little bit of information from what one knows about human cognition but then just saying, these are the design rules; now figure out how to create an algorithm that will produce visual output that satisfies these design rules. At that level, you can ask a question like, when you type in some quantity, like 235 inches. The question is what you want to know, based on 235 inches. You probably want that converted to feet, you might want that converted to meters, you might not particularly care about having that converted to microns, for instance. There's the question of how do you figure out what humans would like to see that converted to. In that particular case, the way we figured that out for lots of different kinds of units is we looked at the Web and we looked at scholarly articles and things like this, and we just crunched the whole thing, and we looked at when people talk about inches, meters, centimeters, microns, nanometers, things like that, do they say 10 million nanometers? Probably not. Do they say 230 nanometers? Absolutely. Turns out there's a distribution you can actually deduce from looking at that data set, which is the Web, about how people talk about things like units.
I would think in a case like 235 inches, how likely is it that people would just type that into a computation engine like yours? I would think it's more natural to say "235 inches in meters" or something like that. It's shocking that they don't go that extra step further.
Not the case. They don't. Because what happens, if you're thinking about it—so you figure out something is 235 inches. Then you say, What really is that? So you type it in, and we'll tell you it's some number in feet, meters, centimeters or yards, and they say, That's what I needed to know.
So putting it in meters is beyond their understanding of what they want to know.
For example, let's say they were reading an article in Popular Science, and you had something that said such-and-such a thing was some size, some number of liters or something. And the person says, Well, what is that? Then they type it in, click a link, and it says that's actually also 22.3 gallons or something. Well, maybe we also generate these comparisons something-something-times the actual size of a Coke bottle. And now they say, Oh, now I understand, on I go, I can do something with that. This is relating the raw data to the kind of map of knowledge that we humans actually have. The real thing is how you go from real data to things that are useful for humans, useful things that humans can assimilate. One piece of that is you have to actually compute things from the raw data. Another thing is you have to deliver the result of those computations that is sort of well matched to humans.
Sometimes you just have to try it, and then you will know.
How many tries will it take for a robot to do a kickflip?
Wolfram Alpha says:
Let's see it happen!