GreenBkk Tech | Humans vs. automated search: Why people power is cool again

Humans vs. automated search: Why people power is cool again

By Pete Cashmore, Special to CNN

Pete Cashmore thinks human-powered decision-making may increase the quality of search results.

Editor's note: Pete Cashmore is founder and CEO of Mashable, a popular blog about social media. He writes a weekly column about social networking and tech for CNN.com.

(CNN) -- What's the best way to find great links on the web? Is it algorithmic search engines like Google, people-powered decision-making, or a combination of both?

This has been the perennial question online ever since Yahoo's human-organized directory of web links lost the battle to Google's automated crawlers. And yet now, in some circles at least, the pendulum appears to be swinging back: Google is broken, say some technologists, and people-power is the best fix.

What's the problem with Google search? Spam, say the critics. Or rather, a new genre of not-quite-spam. So-called "content farms" have sprung up to produce thousands of articles per day with titles that match popular search terms -- dig in to the content however, and you'll find it anything but useful.

This Google-fatigue began as early as December 2009, when venture capitalist Paul Kedrosky remarked that using Google to search for a new dishwasher delivered nothing but spam results.

"Info-krill", Kedrosky called it, "Identify some words that show up in profitable searches -- from appliances, to mesothelioma suits, to kayak lessons -- churn out content cheaply and regularly, and you're done."

It's a concept that was revisited this month by prominent British blogger Alan Patrick, who remarked: "This year it really hit home just how badly Google's systems have been spammed, as typically anything on Page 1 of the search results was some form of SEO spam".

Others have come to the same conclusion. Prominent software developer Jeff Atwood this month declared "Trouble in The House of Google", while entrepreneur Vivek Wadhwa outlined his reasons "Why We Desperately Need a New (and Better) Google", namely "spammers and marketers". Developer Marco Arment, meanwhile, decried the "apparent explosion recently of cheap-'content' sites", which have contributed to "Google's apparent defeat by spam."

While the problem is clear, do technologists agree on the solution? It appears they do. Atwood asks: "Are we seeing the first signs that algorithmic search has failed as a strategy? Is the next generation of search destined to be less algorithmic and more social?"

Kedrosky too proposes a human-powered solution, declaring that "curation is the new search."

Wadhwa, meanwhile, points to new search engine contender Blekko, which searches within sets of websites defined by human editors.

And Arment proposes a combined approach: "One solution may be for Google to radically change their algorithms and policies for web search to de-emphasize phrase-matching and more strongly prioritize inbound links and credibility. And, in what's probably a huge departure for them, have human employees use their opinions of site quality to manually adjust the relevance of domains."

While I'm inclined to agree that human-powered decision-making may increase the quality of search results, it's worth noting that the path to "curation" is littered with bodies.

In December 2010, news leaked that Yahoo was looking to close down the social-bookmarking site Delicious. Yahoo later clarified that the company was instead looking to sell the service, which had pioneered wide-scale, people-powered categorization of web links. Yahoo's attempts to jettison the site are hardly a shining example of the rise of human curation.

Last year also saw the downfall of another Web 2.0 pioneer -- Digg.com, which enables users to vote on Web links and ranks the most popular results, saw its web traffic plummet after a redesign that aimed to keep pace with Twitter and Facebook.

It's there, I think, where we might finally see curation bloom: On Twitter and Facebook, where users share millions of links per day without even the mildest sense that they're "curators" of the web.

To these everyday internet users, they're just sharing cool websites with friends; To those who seek to make sense of an increasingly complex web, these millions of social signals are gold dust.

The solutions will come in many forms, from Bing's integration of Facebook "Likes" into its search results to Twitter- and Facebook-powered newspapers like Paper.li and The Twitter Tim.es. But ultimately, and with a few stumbles, we'll find numerous ways to combine human judgment with the efficiency of algorithms.

Credit: CNN (www.cnn.com)