Searching for Answers to Life’s Persistent Questions

8 March 2009 by Sanjay Saigal.

I recently mentioned LaserSearch, an engine for user-guided web search. Now comes the news that Steve Wolfram, among other things the inventor of Mathematica and author of A New Kind of Science (NKS), has also devised something new: a computation-driven semantically-aware knowledge search engine called Wolfram Alpha. (Was Alpha Wolfram taken?)

Here is Wolfram’s description of his approach:

… what about all the actual knowledge that we as humans have accumulated?

A lot of it is now on the web—in billions of pages of text. And with search engines, we can very efficiently search for specific terms and phrases in that text.

But we can’t compute from that. And in effect, we can only answer questions that have been literally asked before. We can look things up, but we can’t figure anything new out.

So how can we deal with that? Well, some people have thought the way forward must be to somehow automatically understand the natural language that exists on the web. Perhaps getting the web semantically tagged to make that easier.

But armed with Mathematica and NKS I realized there’s another way: explicitly implement methods and models, as algorithms, and explicitly curate all data so that it is immediately computable.

It’s not easy to do this. Every different kind of method and model—and data—has its own special features and character. But with a mixture of Mathematica and NKS automation, and a lot of human experts, I’m happy to say that we’ve gotten a very long way.

As with everything Wolfram touches, there is much that intrigues. Let’s suppose that Wolfram Alpha can answer a question of the following form: in which US counties did the Libertarian Party get more than 1% of the vote in the 2008 election cycle? If so, can it also compute for you the covariance of the Libertarian party’s electoral success (so to speak) with the prevalence of advanced degrees? And if it can do that, can it automatically estimate Libertarian votes in 2012 based on country demographic forecasts? Etc.

Taking the above process to its limit, can Wolfram Alpha – say, sometime in the distant future – become an all-purpose web-based Analytics engine?

Posted in Business Intelligence | 1 Comment »