In a final coda to the sad saga of Wikia Search (the overhyped Wikipedia-model user-generated free labor search startup, which crashed and burned into being a Yahoo front-end before imploding completely), the site http://search.wikia.com/ now leads to nothing but Wikia's copycat wiki answers site. There's a little section on the homepage there:
The Wikia Search project has ended. search.wikia.com redirects here. Find out more:
* What was Wikia Search?
* What happened to Wikia Search?
* Where can I get the source code for Wikia Search?
I have nine printouts and lead sheets, mostly related to Wikia Search, Wikia's odd effort to take on Google by crowdsourcing search ranking itself. But I notice an oddity similar to the CZ cluster, and maybe it shouldn't be surprising: All the items are from a single blog, in this case Seth Finkelstein's InfoThought ... Searching that blog for "Wikia search" yields a lot of results; he's covered it in depth.
Why is all my Wikia Search stuff from one source? Maybe because, despite lots of praise when Wales started talking up the idea in 2007, the reality has been... tepid. When the public availability began in January 2008, SearchEngineLand called it "really just yet another crappy search service." The more you read of the whole basic idea, the less it seems to make much sense in the real world.
Sadly, there's a market for hype, but not for skepticism.
"The shutdown of Wikia Search - an attempt to apply Wikipedia-style ideas to create an open source, commercial search engine - came as no surprise to informed observers"
I didn't suggest a title, leaving to the editors what they wanted to emphasize, and this one is fine.
I have some good turns of phrase in this article - calling Google the "Great Search Satan" (looks like nobody ever used that phrasing before), and talking about "mining user-generated discontent". And I close with "a taint of half-baked dilettantism and corporate exploitation.".
An open search engine is a great idea, and I support the concept in general. But Wikia had too much of a conflict of interest in term of corporate motives to be right for it.
Blog bonus: I wanted to use this old joke I've mutated, somewhere, but it was too long for the column:
Two entrepreneurs are in the woods following the trail of a ferocious predatory Google. All of a sudden, the Google crashes out of the brush and heads straight for them. They scramble up the nearest tree, but the Google starts climbing up the tree after them. The first entrepreneur starts taking off his heavy leather hiking boots and pulls a pair of sleek running shoes from his backpack. The second entrepreneur gives him a puzzled look and says, "What in the world are you doing?"
He replies, "I figure when the Google gets close to us, we'll jump down and make a run for it."
The second guy says, "Are you crazy? We both know you can't outrun a full-grown Google."
The first one replies: "I don't have to outrun the Google, I only have to outrun you!"
[Wikia Search did not have to be a "Google Killer". It only had to outrun its costs, using digital sharecroppers as bait, to make a profit (though it couldn't even do that)]
Wikia Search dead, says CNET:
Wikia is announcing on Tuesday that it is closing the Wikia Search product. The service was intended to be a user-generated search engine, through which users could influence the rankings of results for all other users.
The Wikia Search project is set to be shut down Tuesday.
Corroborated by a post to the moribund search-l mailing list:
From: "Mark (Markie)"
Subject: [Search-l] The end is here...
http://news.cnet.com/8301-17939_109-10207896-2.html tells it all.
it was nice working with you all, thoughts go to all the staff who have been let go because of this. who knows, we may meet again the in the future at some point.
ps: http://search.isc.org/ -> go there :)
I had seen indications of this last month, but I'm not exactly their favorite person, so they likely weren't going to talk to me.
[Techmeme readers: You may enjoy my earlier column on Wikia Search
"When you have a Wikipedia, everything looks like an edit"
They-said-it-not-me: I like the way VentureBeat describes the PR
In confirming the news, Wales attempts to deflect the bad news by pointing to the success Wikia.com, Wikia Search’s parent, has seen in terms of growth over the past two years. But towards the end of his post, he concedes that Wikia Search has not had the success that the company had hoped for.
I recently noted the Wikia Search using Yahoo operational change, speculating:
"And one wouldn't want people who go to Wikia Search currently to think the results are proof of anything other than that Yahoo has a program to allow others to use its search system. It would be pretty easy to get a misimpression along the way."
It turns out at least one person got exactly that sort of misimpression:
"... I really think this Wikia search has the ability to beat Google in some key areas. I've already discovered a few searches that Wikia Search beats Google on, and I figured I'd write one down - dreamhost wiki."
But it wasn't "Wikia Search", it was Yahoo's search.
I wonder if any reporters will be similarly fooled, and write even more Google-killer hype articles. Certainly I don't see any "Powered by Yahoo" identifier on Wikia Search now. For a project that touts "Transparency" as a main goal, that's rather ironic.
I checked the Yahoo! BOSS (Build your Own Search Service) details, and they don't require attribution, so what Wikia is doing is permitted. Still, given all the free publicity Wikia garnered, along with the storyline of killing Google (a complicated matter, something of a media invention), having it silently being so much a rebadged Yahoo! Search seems like something which, morally, should be more evident. The phrase "legal, but sleazy and unethical" comes to mind.
Disclaimer: I'm a member of the "negative people and FUD mongers".
Wikia Search, the project to buy Jimmy Wales a jet, I mean, have a Wikipedia-like search engine (i.e. unpaid labor does the work), changed over at some point from using their own back-end to using Yahoo! BOSS (Build your Own Search Service). While not a secret, it's been difficult to find information about the switch (I believe for obvious reasons ...).
Today in a discussion thread Wales posted about the change:
In order to get the front end going successfully (which we have, although there are a lot of bugs to fix and a lot of unmet community needs so far), we needed to have a "usable" algorithmic search.
But our own crawl+algorithm was not yet good enough. So usage wasn't growing, it wasn't possible to "eat our own dogfood" even.
So we have to currently use Yahoo! BOSS for the backend (which is actually pretty decent) to build traffic and participation, while continuing to work and figure out the backend for the future.
Good to have it on record. And one wouldn't want people who go to Wikia Search currently to think the results are proof of anything other than that Yahoo has a program to allow others to use its search system. It would be pretty easy to get a misimpression along the way.
I'll note he goes on to say:
I would totally ask that we switch back to index.isc.org, or to any other free-software solution, as soon as it is "decent enough" - doesn't even have to be *better than* Yahoo BOSS.
Yes, but the difficulty is that people kept pointing out that this was a very hard problem. And Wikia, a venture-capital funded start-up company, was effectively getting a large chunk of its research and development costs here subsidized and funded by user donations. Many search start-ups would love publicity and practical subsidy like that.
Hmm, powered by Yahoo for the moment, selling Google ads, feel how this is really an endeavor to fight the power and bring democracy to search (stand up and cheer! buy The Founder a jet!).
Disclaimer: I'm an idiot.
Wikia Search just posted about turning blog posts into a "Wikia Search Application". That has the potential to be amazingly contentious, and dangerous - if anybody cared! Basically, if Wikia Search actually had any users, and so was worth going to the effort of fighting over spaces and scamming people, that feature would be a trouble-magnet. However, "luckily", essentially nobody uses them, so it's not going to cause much of a stir.
In sum, they're allowing blogs to be "Search Extensions" for keywords. You can nominate any blog for any term. Subject to their approval. Now, I happen to think my blog is quite relevant to terms like "Jimmy Wales" and "Wikipedia" and "Wikia" ! :-) Would they let me in? For those keywords? I can think of several other folks with blogs who would also love prime ranking for those terms.
Someone could object that this is the same problem with as Wikipedia, and that was sort-of addressed, enough to be functional (with lots of cost-shifting and misery). But it's often said, search is worth real money, and that's orders of magnitude more difficult to handle.
Anyway, I'm not going to bother trying to get accepted as a bona-fide "WISE"-guy. It doesn't seem worth the potential backlash. But I wish coverage pointed out these problems more often.
Wikia search is releasing "Wikia Intelligent Search Extensions" ("WISE"):
WISE -- "Wikia Intelligent Search Extensions" -- is Wikia Search's application framework. WISE makes it easy for website owners and developers to provide Wikia Search users with search results enhanced by the data available across the Web.
There is a problem with that paragraph, in the words "Wikia Search users". To a good appproximation, there aren't any. It's a long way from buying Jimbo his jet.
In a way, it's pretty funny. Wikia Search has no traffic and no quality results. Therefore, ask partners to provide the quality results - in the guise of reaching the audience that isn't there! I've got to say, it's a neat search engine twist on the strategy of digital sharecropping sites.
Note the press release contains this revealing portion:
"Here at Last.fm we consider Wikipedia to be one of the most pioneering community-driven websites of the last 10 years, so we're thrilled to be a part of Jimmy's new venture," said Martin Stiksel, Last.fm co-founder.
I don't make this up. There's the benefit in a nutshell that Wikia the for-profit venture capital backed startup company obtains from Wikipedia the famous non-profit site.
"It's informative to observe how long Wales has been pursuing a strategy of selling advertising around other people's work."
[Checking, my blog post which used to rank highly on a search for his name has been user-deleted from those Wikia Search results (it's greyed-out a bit down the page as a user-deleted item). But those search results are currently full of "Valleywag.com" posts, which he may not consider an improvement]
For example, when launching and running Mahalo, everyone wants me to say we're a Google killer. When that question comes up I immediately say "let me be very, very clear -- and please don't misrepresent me in your story -- that Google is a partner and we don't see ourselves as a replacement for Google but rather a complement for the following reasons." ...
Your job as a suspect/subject is to say things concisely and with few words: "Google is our partner in five areas already: search advertising, analytics, YouTube, open social and custom search. They also send us half our traffic--they are NOT our competition, they are our partner." Silence. More silence. If the journalist is good they will say something like, "but certainly on some level you compete?" and you respond "No, we don't."
Of course, the incentives are different for Mahalo than Wikia Search. Mahalo really is a partner with Google. While Wikia Search, well, it seems absurd to say it could replace Google, more like a roach trying to eat a scrap from Google's meals (competitor only in the most abstract and technical sense).
When I said "I'm not worthy", it looks like someone took me at my word :-). Last week I noted an example of one of Wikia Search's problems, in that a pure equal-weight popularity voting system tends to generate buzzy results - giving one of my own blog posts as an example. Someone seemed to have taken exception, and then voted that post down in the search results, among others. I think someone else voted it back up a little later (just what's been missing from search engines, right, edit-wars).
But I'd say there's actually been a notably small amount of participation in the Wikia search engine. It gets an enormous amount of PR, which is a functional subsidy that other search engine start-ups can only envy. But so far, not a huge amount of free labor.
All the blather about taking on Google is so over hyped it's painful.
I'm not worthy:
For those who can't see the image, my Wikia Search Follies blog post is the second result for a Wikia Search for [Jimmy Wales]. The first result is currently an article from The Register, "Jimbo Wales dumps lover on Wikipedia" (n.b., though that makes for an ironic story, I suspect it really didn't happen that way).
In Wikipedia, functionally, articles are effectively controlled by a small clique (forget all that blather about whizzing crowds, it's nonsense). Wikia Search hasn't figured out how to do that for search results. Frankly, I'm surprised the results for a search for [Jimmy Wales] are relatively inoffensive. But then, the net results are basically an approval-voting system, which favors scandal over pure trolling. So there's a lesson there - they may end up not making a search engine, but a selection game seeded by (bad) search engine results.
I noticed an interesting little item in the release of Wikia Search's "Evolution" browser add-on (it's designed to hand-scrape Google and Yahoo results, but others, with much greater audience, have already chewed that over - this post is value added). The owner data is:
Wikia Search Evolution 0.1.0
by Wikia Inc (Ashish Datta)
As I write this, the addon has three "reviews" and a rating of five stars (out of five). The third so-called "review" is by Jimmy Wales (only four stars :-)), and is really a plug. But this isn't even about that obvious conflict-of-interest minor hype. No, what's the first review?
[Five stars] Wikia Evolution!!!
Great add-on! Great search engine!
by Set Five on August 1, 2008
Now, who is user "Set Five" ?
Firefox Add-ons user since
August 1, 2008
"ashish"? Where have I seen that name before? Oh, right, "Ashish Datta". Who, according to LinkedIn, is "CTO at Setfive LLC" (and "Jester at Wikia").
You know what the Wikipedia apparatchik class does to people who do stuff like that on Wikipedia ...
Very interesting article here about Jimmy Wales, Wikipedia, Wikia Search, and some Google-politics:
Economist: "The free-knowledge fundamentalist".
Yes, I've said this before, but bear with me. I get much flak for my views. Even if this is a tale told by an idiot, here's another case of where, in the business press, there's interesting nuggets and not just chump-fodder:
By the late 1990s, Mr Wales was investing in a website called Bomis, a sort of search engine or web directory where "99% of the searches had to do with naked babes," as Mr Foote, who was Bomis's advertising director, puts it.
Hee hee. Remember, they said it, not me! And money, money, money:
As Mr Wales struggles with such intellectual controversies, he now does so as a minor celebrity. Neither Bomis nor Wikipedia has made him rich -- if he is comfortable, it is mainly the result of earning money from speaking engagements, say friends. ...
All this has gone to his head, say former friends. Mr Wales "has created something of a mythology about himself," says one. "The image he created is that he is this benevolent millionaire who donates his time for this charitable project; that is not true." Instead, this acquaintance argues, Mr Wales is merely basking in the glow of Wikipedia's success. ...
Again, they said it, not me! And remember what fuels the value of those speaking engagements. That's you-Yes-YOU! Now, don't you want to buy him a jet too? As in:
So far Wikia's search results are embarrassingly poor, as reviewers have noted. And there are more fundamental doubts. Wikipedia succeeded because, in 2001, there was no free online encyclopedia. Today web search, by contrast, is a hyper-competitive industry. Mr Wales does not see it that way. Search has become a window to knowledge, and Google and its rivals have become its arbiters. "For me it's a political statement," he says. "We don't need secrecy. ...
See, you're supposed to work for free to build a search engine for Wikia to monetize, as a political statement. Against the arbiters, which are cast in a role of the new Evil Empire (which now means Google and its rivals, not Microsoft). And it's a bad rebel who wonders how much he or she gets paid for doing this (and further ponders how the grand rebellion will benefit the glorious leader's bank account).
I'm late to the party in blathering about the latest updates to "Wikia Search" (the attempt to Wikipedia-ize search with free labor). And it's not like my blog post is going to be much read anyway. So I'll just refer to my column last month which discussed the topic extensively:
"But the idea that these simple systems can be applied to deep value-laden social problems, of politics, or even relevant search results, is like trying to use a hammer to turn screws on the basis that it works so well to hit nails."
"The Jimmy Wales Experience" was just published in a financial press organ called "Trader Monthly Magazine". I find these sorts of articles very interesting, since they have a perspective vastly different from the Kool-Aid doled out for the consumption of the rubes. And note if I said the exact same things as appear here, I'd probably get intensely personally attacked as a negative person. The article requires site registration, so I'll share the best parts. Like this gem:
In 2004, he launched another Web site, this one called Wikia Inc. (See "Wikipedia 2.0.") The company, Wales readily admits, is his effort to take the success -- and, indeed, the underlying philosophy -- of Wikipedia, and commercialize the hell out of it. "Look, I'm not against making money," he says.
[But remember folks, it's all about sharing. It's about the community.]
He decided that there was no reason he couldn't become the Internet's Michael Dell. In his spare time, he experimented with a handful of Web ventures, like a kid in a young entrepreneurs' club. He started something called Loop Lunch, a site where office workers in downtown Chicago could order food online from local eateries. It flopped.
In 1998, at the height of the dot-com gold rush, Wales resolved to go for broke. He quit the firm, took his savings and left for California to take part in the boom.
[But he's always been about bringing knowledge to the people.]
Wales, meanwhile, has gone on to fame, if not exactly the enormous fortune one typically associates with Internet moguls. But it's not as if he's opposed to rectifying that situation. Early in Wikipedia's life, Wales and his partners considered selling ads on the encyclopedia's pages. The site was showing signs of explosive growth, and they certainly could have used the extra money. Though ultimately they nixed the proposal (Internet ad rates had fallen off anyway, of course), they didn't exactly do so for idealistic reasons. "We've never said, 'Absolutely not, we don't want to sell ads,'" Wales says, explaining that the decision had more to do with preserving the Wikipedia brand.
And with the advent of his for-profit venture, Wikia Inc., it appears Wales is finally ready to monetize.
[They said it, not me!]
The company's business plan maintains that the bulk of its revenue will be generated by...drumroll, please...advertising.
"The monetizing is pretty straightforward," Wales says. "We don't have any clever, innovative ideas around that."
But even after three years of operation, the site has refrained from selling ad space, content to build critical mass before it goes full-tilt with its sales effort. For now, it's about brand building.
Wales's ambitions for Wikia don't stop with online community-building. At the end of 2007, his company launched an alpha version of Wikia Search, with which Wales aims to do battle with Google. ... Instead of search results produced solely by computer programs, the Wikia engine will use its wiki-based communities -- and the actual human beings who participate in them -- to refine its results. The hope is that these people -- the open market -- will edit out the spam and other extraneous junk, producing a series of links that Wales believes will be more relevant to people than what Google generates.
[Saying "the open market" sounds so much better than "the unpaid masses" (at least Mahalo pays the piece-workers something). But isn't it fun to buy Jimmy Wales a jet?]
Regarding the list of questions on Wikia Search, Jimmy Wales wrote a reply. I'll refrain from summarizing his responses, as whatever I said could be attacked as biased and unfair. Anyone interested can read the whole thing.
I'll highlight this part of the exchange, without further comment:
Paul Vixie: "[T]he noncredibility of the early claims about "taking on google" is the biggest weakness wikia search has got."
Jimmy Wales : "And this meme (which comes mostly from the media that loves a David and Goliath story) is also probably the biggest strength we have got right now. It's a monster either way. :)"
Paul Vixie: "[T]he truth is stronger. i recommend running with it and abandoning the meme."
1. why is ISC's the only backend?
2. why is Wikia's the only frontend?
3. who is driving the syndication model?
4. what else is [the project lead] working on?
5. who else is working on this, outside of wikia?
6. where are the mini-articles stored?
7. ... where are the white papers, journal articles, and outreach glossies explaining what the new world of internet search could look like ...
8. has anybody reached out to yahoo and microsoft ...
He said these, not me :-).
Jimmy Wales wrote back that he'd respond, so we'll see.
[Update: Jimmy Wales replies ]
The Wikia Search project (Wikipedia-model search engine) mailing list has had a thread discussing the status of updates and the social organization of the project, and which sort of arrangement would be best for the goals of having an open search project. There's been some confusion between who owns what part of the current technical infrastructure. Paul Vixie, who runs the Internet Systems Consortium (ISC) clarified (comment reformatted for readability):
The servers were donated to ISC, not Wikia. The bandwidth they use is provided by ISC. Wikia has donated a 15-ton air conditioner, a smattering of network switches and front end servers, and a heaping lot of [Wikia search lead]'s time and other Wikia staff time. Wikia has agreed in principal to underwrite the power costs of the ISC physical plant used by the crawlers and indexers. But as for a [different organization] there already is one (see www.isc.org, ugly though it is) and i think it's odd that anybody is still worried about that part. ISC is a 501(c)(3) [nonprofit] whose mission is public benefit. If anybody here thinks we're either incompetent or untrustworthy with regard to owning and operating a search engine backend, I'd thank you very much to call me on the phone and explain your concerns to me realtime.
My view is that the problem is not ISC's bona-fides. Rather, it's more the issue of Wikia's incentives as a VC-backed startup, versus the optimum structure of an open-source search project. Focusing on Paul Vixie or ISC shifts towards what I call the "positive" ad-hominem argument (roughly: "I'm a good person. Therefore, what I do must be good. If you say it's bad, you're saying I'm bad person. Because only a bad person does bad things. But I'm a good person, and you are then a bad person for saying otherwise. Therefore, you must be wrong. Because I'm a good person").
I'm a strong supporter of the ideas of Free Software and Open Source. But Wikia seems to be doing the worst implementation of this kind of project. Which is the arrangement where supposedly programmers do unpaid labor because they just luvvvv programming, and businesses make money off the honey from the little worker-bees.
Semi-related, there's an amusing graph of the Wikia Search Hype Cycle
Are you going to help take it to the next level?
Hmmm ...that's "you", as in you-Yes-YOU? No. I don't think so. I'm very wary of the way Wikia is positioned to "brand" everyone's work as its own (after all, that's happening right now with ISC's servers), and to commercialize itself anything of value from independent developers.
Building an editable encyclopedia is nothing compared with the challenge of building a search engine that can take on Google
Note the front of the Guardian site has it as "Wikia Search is doomed to fail", which wouldn't be my preferred title or summary (I didn't write either of them), though they're not wrong either. I know it's conceivable that Wikia Search might not "fail" in terms of producing something that Wikia can sell (perhaps to Google, Microsoft, or Yahoo), or even turn a profit - especially given how many expenses they've shifted off to others. I'm sure they'd count that as a "success" (and it's all due to you-Yes-YOU!). But I don't see any prospect for it having a deep effect on the field of search (the field of getting people to work for free is a different matter).
The idea I'd like people to take away though, which I did write, is this:
"But the idea that these simple systems can be applied to deep value-laden social problems, of politics, or even relevant search results, is like trying to use a hammer to turn screws on the basis that it works so well to hit nails."
[For all columns, see the page Seth Finkelstein | guardian.co.uk.]
[Post updated since the front site title is different from the article title]
There's a Wikia Search puff piece in the Los Angeles Times: Wikia's Weird Dream: "A new search engine from the minds behind Wikipedia relies on the human touch to outsmart Google. Does it stand a chance?"
It won't do any good for me to rant about this to my tiny audience, but I'll do it anyway in vain hope it'll show some of my critics that the hype I'm talking about does indeed exist:
Wikia, working from modest headquarters in San Mateo, Calif., is attempting something at least as audacious as Wikipedia's launch was in 2001: It is building a for-profit search engine to compete with Google. There are no legions of engineers. Instead, the endeavor is powered as nonprofit Wikipedia is: by volunteers. (Wikia and Wikipedia are different companies; all they have in common, besides a "by the people and for the people" philosophy, is their co-founder, Wales.)
Note how Wikia is given a halo effect from Wikipedia - "by the people and for the people" is much more accurately rendered "by the unpaid and for the investors". And later on, the answer as to how this sales-pitch works:
Why would volunteers donate their time to help a for-profit company?
"Wikia Search is basically doing something unique," says Mark Williams, an 18-year-old college student who lives on the south coast of England. He estimates that he puts in up to 20 hours a week rating search results for Wikia and writing mini-articles. "It's changing the future of how people can search, so that they know how and why certain results are coming up, and if they don't like the results then they can say so."
You - yes, YOU - can say so! And if your contribution helps a corporation make more money, then you can have a warm and fuzzy feeling from it. Unemployment insurance? Health benefits? A union? How can these compare to changing the future? (a semi-serious question).
Observer that the guy apparently hasn't heard of any of all the other social-search engines. That's the value to Wikia of all the publicity from Wikipedia (not the tiny bits of PageRank that might be sent to some Wikia sites from any hypothetical Wikipedia favoritism towards Wikia, or even various personnel connections).
Wikia executives have another criticism of Google. They look at its inability to control spam from its e-mail service, Gmail, and they see a parallel with the search engine: Google has been unable to entirely eliminate irrelevant search results. "I'm constantly surprised by how much money, how many computers, how many algorithms go into preventing spam, and yet every day I look in my mailbox and I go, -Dude, that's not right,' " Penchina says. What's striking, he says, is how much better people can do than Google's software in filtering out unwanted e-mail. In the same way, he says, volunteers can do that with search results.
Why, yes, if you can get a large pool of uncompensated labor to do lots of scut-work, that would work very well. The trick is, of course, getting that in the first place.
Meanwhile, all the Wikia-bashing has also resulted in some goodwill, even sympathy, from critics. Sherman of Searchengineland was critical of Wikia Search's launch, but he says he would like to see the underdog pull off an upset. "I'm kind of rooting for them because of the David versus Goliath aspect," he says. "Google has started taking on aspects of the Microsoft evil empire - it's too big, too dangerous. And isn't it cool that we're going to have an upstart that will give us an alternative?"
Volunteer for us to fight the Evil Empire! (pay no attention to the money we'd like to make off of you - you're changing the future, you're part of The Revolution!)
Others warn it would be a mistake to underestimate Wikia Search based on its early performance. "Anyone who is selling Jimmy Wales short over the launch is going to be in for a surprise," says John Palfrey of the Berkman Center for Internet & Society at Harvard Law School. "He's leveraging many of the same things that made Wikipedia a global force. I think Wikia can have a huge impact on search engines over time."
And here is again why I will never be an insider, a club-member. I'd concur that Wikia Search is *trying* to leverage many of the same things as Wikipedia - but in my view that's not necessarily good.
I continue to be unimpressed by Wikia Search, the Wikipedia-model search engine. They've released a bunch of new "social" features for editing and annotating specific results, which are not in and of themselves bad things. Yet I keep wanting to say, but, but, but, there's no real search engine there. Nobody gushing over all the pretty buzzword-compliant aspects seems to care that when putting lipstick on a pig, underneath, it's still a pig. Maybe they think a sufficient amount of lipstick on the pig emergently create a useful search engine.
As I understood the initial Wikia Search concept, volunteers were supposed to build Jimmy Wales a search engine for free, that the venture-capital backed start-up Wikia Inc. could then monetize with ads (or presumably flip in a sale if the opportunity arose), because this would then prove The People open-source amateurs can challenge the closed proprietary Google elitists. Or something like that - obviously it wasn't stated so bluntly.
A major problem here is that search engines require very specialized expertise, for which companies are willing to pay big money, so almost nobody wants to give it away for the good of Wales's gold-plated washing machine, I mean, humanity.
But the latest iteration of Wikia Search seems to be trying to use the very poor search technology as a seed page for human-edited results. That is, an unpaid, voting-driven, Mahalo.com. I suppose if the workers aren't paid, anything at all is profit. But I can't help but think this is turning into a proof of my theory that Wikipedia is basically a weird thing, which doesn't export the secret of the fountain of free labor.
Puff piece: "Wikipedia founder's next target: Google"
"It's no longer super rocket science that only the geniuses at Google know how to do," said Wales ...
Maybe, but there's no evidence that Wikia knows how to do anything beyond fizzling out. The following is not an original observation, but worth remembering: One important "community" difference between Wikia Search and Wikipedia is that there are no hot "encyclopedia start-ups", and markets for expertise in "Encyclopedia Optimization" (well, the latter job is arguably "reference librarian", but it doesn't pay nearly as well as being a search engine optimization expert)
More interesting is this little nuggest (my emphasis):
Wikia is moving to San Francisco, where most of its 50 employees live, from San Mateo, California, about 20 miles south, Wales said in the April 8 interview. The office will be around the corner from Wikimedia Foundation Inc., the non-profit parent of Wikipedia, ...
Just fortuitous, I'm sure. It wouldn't be "assuming good faith" to think anything more.
Wikia Search, the project to bring the Wikipedia-magic of unpaid labor, I mean, web collaboration, to search engines, has not been heard from recently. A digital-sharecropper of the electronic plantation, err, community member, is restless:
*Whats happening with the project. AFAIK overall (and i know somethings have happened) but *very* little seems to have happened since the launch. ... Many things have been promised by various people, which havent happened. Most specifically this has come from a certain member of staff, one specifically, that has said that they will do many things, but even the most basic of tasks seem to have not happened. so Broken/missed promises. ...
The project lead replies:
I feel and share your frustration, I know at least for my part I've done a poor job keeping going a steady stream of updates on the things I've been playing with. Of course, I've never personally been good at regularity, tend to step back and push things out in spurts... all I can do though is try to catch up for the last month or so of various chaotic activities. ...
First let me say that if anything has been missed, or promised and then not delivered, it was not intentional. Second, I would agree with you that while we have been working to make changes to improve the accuracy of the search results, we have not been doing a very good job of keeping the community informed about those or other changes and that is something we need to work on. ...
Notably to my mind, I hadn't realized that the internal ranking algorithm was so primitive, not even having what's known in public about Google's technology (the launch was atrocious, but that could have just been time pressure). I guess no search experts want to work for free, for the dubious "fun" of making Wales rich.
"Wikia Search tries to draw on the fear and doubt stemming from the dominance of Google"
I've tried to pack a lot into this column, everything from the $50K price for the "Grub" crawler" to pointing out how the politics of search can be used for free labor. I also bent over backwards not to even seem to be using the column to retaliate against Jimmy Wales's conduct, and he ends up only being mentioned in specific for identification (sadly, as far as I've ever seen, it's never done me any good to be morally better my attackers in terms of not abusing power, but I think I read too many comics books as a kid with Good triumphing over Evil - it doesn't work that way in real life).
I've been mulling over the it'll-get-better argument for Wikia Search, which has struck me as problematic:
When he first announced the idea, Wales said the search engine would improve over time, much like the other project he [co]founded, Wikipedia, has.
It's seemed to me that, though it's expected PR, there's something subtly wrong with that comparison, and I believe I've figured out the error.
To wit: For a search engine, a certain level of quality has to be reached for everything before it's usable for anything.
That's stated in a concise way - by "everything" I mean not "every result", but rather that a lot of things *all* have to reach at least a mediocre level: indexing, server response, ranking algorithm, anti-spam, etc. And if any single one of those factors isn't at least passable, the whole search engine is unusable in practice. Roughly, you won't even have a few good results that people in a topic area can use until all the basics are working.
Moreover, it's pretty unrewarding to work on improving internals for free. Where someone might write one Wikipedia article for the joy and happiness of having written it, it's much more difficult to get a volunteer to work for months on a ranking algorithm (note I don't claim it's impossible - students doing a class project or a thesis, or people trying to gain experience, are potential sources of unpaid development - but the free labor pool is much smaller)
So I believe the argument is wrong, in that there's a barrier of functionality, which can't be climbed incrementally with small contributions. Which again is not to say it's completely insurmountable. But that would seem to require an extensive amount of concentrated skilled development.
But then, I am one of the "negative people".
As my Wikia Search troubles would seem to make anything I say suspect, I sat out yesterday's blog-fest reaction to Wikia Search's launch. No need for me to pundit anyway. When TechCrunch on the one side calls Wikia Search "one of the biggest disappointments I've had the displeasure of reviewing" and SearchEngineLand.com says "it's really just yet another crappy search service that may, potentially, if all goes well, eventually turn into something useful", no matter how hard Jimmy Wales tries to spin it by bleating about "alpha"-test, that's a PR disaster. Yes, Wikipedia was minimal at the start, but it wasn't launched after a year of hype about being a Britannica-killer (I know, there was some effort to set expectations about the "Google-killer" story, but still, it was there).
I suppose I should be flattered when in the midst of this debacle for his company, Jimmy Wales took time out of his very busy schedule in order to flame about me again, on the search mailing-list where he's _de facto_ banned me. I was actually surprised. All of this bad press is crashing down on him, and he takes the personal time to make sure he throws some mud at me, where I can't reply. Truly a measure of his character.
I suppose I must resign myself to being one of the, in his words, "negative people and FUD mongers" who thus does not belong in the shiny happy unpaid-labor world he is building on "principles of free culture, transparency, and openness." (!). It's all very cult behavior :-(.
Wikia Search "transparency" doesn't seem to go too far in practice, as Jimmy Wales himself has now basically tossed me from the Search-Wikia mailing list - proclaiming at first "Seth, you're an idiot." and then immediately denouncing my "conspiracy mongering and FUD" (well, I'm technically on "moderation", but given that sort of flaming from The Man, I'd say it's in essence a ban).
Wow. I've long known Jimmy Wales had a problem with lashing out at people who do not trust and love him, and favors sycophantic behavior. But I'm continually surprised at what seems to draw his wrath. It's very much the dark side of the cult guru who has been crossed.
Here's what I wrote which caused him to blow-up, in a little discussion thread about the pending Wikia Search launch (since this is within a thread, there's details which refer to earlier matters, they're probably not important).
Well, it seems obvious that some sort of funding deal got done. My *guess* is that it involved funneling money through the "Internet Systems Consortium". Something like, *hypothetically*, BigMoneyMan was convinced to donate a million bucks or so to the ISC, where the ISC would then run the search project for Wikia. That structuring would mean the BigMoneyMan would get a tax deduction, while Wikia would not be saddled with more venture capital obligations. Then that supports buying many servers and for expert consulting.
I should clarify I'm not saying that the above *hypothetical* is necessarily a bad thing (though small search engines might claim it's favoritism in spirit even if permitted under the letter of the law).
Maybe the deal will be announced Monday, "We got X dollars from Y (via ISC), which allowed us to put Z servers into production and hire ABC". I should stress, before Jimmy flames me for speculation, that under the current "mushroom management", speculation is all I can do. And I assume such a sharp businessman has taken his financial experiences with Wikipedia (as part of a non-profit, "501(c)(3)" foundation) to heart.
There's a large amount of tension between the fact that Wikia is a venture capital backed start-up, and the idea of running a transparent open-source project. Let's remember that the price of the Grub crawler being $50K was not a secret _per se_, eventually being disclosed in SEC documents. But due to the implications, Wikia definitely had an incentive to keep that info hidden as long as possible. Businesses by nature aren't transparent - that's the whole history of the Securities and Exchange Commission in a nutshell.
I suppose I could beg Jimmy Wales to forgive me (err, could have before I did this post). But I'd probably just annoy him again with my irreverence and business-oriented analysis. Oh, and I've got a blog! :-(
[Update: link for memesturbation]
The Great And Powerful Man Behind The Curtain Announceth:
[Search-l] private pre-alpha invites available
Jimmy Wales jwales at wikia.com
Mon Dec 24 03:04:01 UTC 2007
Ping me if you want one.... we're launched. :-)
I'm going to be letting people in slowly over the next few days and we are aiming for a January 7th public launch. We want to run over the system with help from people to complain about what is broken...
Best way to ask is by email, but please don't be offended if I don't answer right away. I am expecting a bit of a flood here.
This is the "Wikipedia-model" search project. Watch the press eat it up, even if now it's only just slightly condensed from vaporware.
[Scoop! News - not an echo!]
A while back, Wikia search, the much-hyped search effort by Jimmy Wales to do a for-profit search engine via a Wikipedia-like model (i.e. free labor from the public), acquired the "Grub" crawler. In terms of following the money, from the Securities and Exchange Commission company filing for the previous owner (LookSmart), it's been disclosed that Wikia bought the "Grub" crawler for $50K:
The URL for Grub and certain source code were sold for $50,000 on July 12, 2007 A gain on sale of assets of $50,000 was recorded in the three months ended September 30, 2007, which is included in other operating income (loss), net.
In January 2003, the Company acquired intellectual property rights from Grub, Inc. and an individual for total consideration of $1.3 million, consisting of $0.6 million cash payment, including $12,000 of direct costs and the issuance of 217,000 shares of LookSmart common stock valued at $0.7 million.
Now, the $50K price might be connected to the Wikia / LookSMart ad deal, so it may not be the full story in itself. But there's definitely a money lesson in here somewhere, even if it's not clear what it is.
There's a brief interview with Jeremie Millier about the current status of what he's doing for "Wikia search", which is the for-profit Wikipedia-model search project.
I'd submitted a few suggested questions for this interview, but they were all rejected. I had wanted to know:
1) Roughly, how many people will be *paid* on the project?
1b) Can you specify whether at developed vs. developing economy pay scales?
2) Do you plan to hire anyone with search engine development expertise?
3) Do you think there's a cultural conflict between Wikipedia's model of operating, where in theory nobody owns any articles, and code development, where typically specific people "own" various subsystems? Which path do you plan to try to follow?
Note understanding 1b) requires some context. It was based on how the company Wikia had decided to offshore programmers - to Poland! That's definitely not something that's talked about a lot.
"Wales said he settled on Poland in part because software engineers there are simultaneously highly skilled and affordable, a combination that he said is hard to find, even elsewhere in Eastern Europe."
[Keep in mind, all you US programmers who are tempted to fall for the marketing, you're not affordable - everyone thinks it's going to be the other guy who works for free.]
Anyway, even though the interview only covers technical topics, it's still worth a read if you're interested in some details of what's behind the hype the audience is being fed.
Interesting interview. I didn't realise that Grub was quite that bad.
On the search side, the Wikiasearch project (if we can call it that) doesn't seem to be doing anything beyond what hundreds of small search startups are doing. The management, bundling and repackaging aspects is, so far, perhaps the only innovative angles.
Now Wales plans to apply Wikipedia's experience to internet search. In December, he will launch Wikia Search, a search engine to compete with giants such as Google and Yahoo. The project could prove even more controversial than Wikipedia.
As value-add, I'll note what wags on the relevant mailing list said about this - note they said it, not me!
[Doesn't say which December ;-)]
True. Nearly a year after the shock and awe of the press barrage started, there is finally a release date. Now if only there was a product. Wikia could be the next Microsoft... ;)
But Microsoft doesn't expect you to build their products for free ...
Wikia CEO on lunacy, air miles and being profitless (my emphasis below)
Penchina: People say to me, "That's crazy, this is something you spend a million dollars putting together and probably a half a million dollars a year updating, and you're just going to give it away for free?"
Actually, we might spend a quarter million dollars this year on the whole search project, on the engineers we pay, the search we run. ... But by having users help out with this, the cost of crawling the Web is close to zero. There are some storage and bandwidth costs, but some partners are providing storage and bandwidth for free. [He declines to name them, citing nondisclosure agreements]
Between the crawl being free, plus the bandwidth and storage, it's pretty cheap. Plus, by offering it out to the community, anyone else developing their own search engine will feed back software, patches, etc., which will strengthen the overall project. This is a mission our users believe in.
The philosophical tenets are so in tune with what we were doing anyway -- volunteerism, free information, free software, making the world slightly better in some way -- that we sort of had to do it.
In other words, you volunteer to make the Wikia investors rich. Because it's "community".
The project has some very unpleasant overtones of a businessman's fantasy of what Open Source means - all the user-gnomes work for nothing in the coding mines, grubbing out gold for the King. Because golly, it's fun.
But I'm not on any crusade to Save The Programmers. It's not necessary. For example:
Search is about turning hard data into information that people can use. Forgive me if this sounds a bit cynical but it has been a long day. Right now, I think that the Wikia people might have been hitting the koolaid a bit too hard. From the whole "user" angle to this happy-clappy nonsense, it doesn't seem like Wikia has much of a plan.
We in the real world of search deal in data and distilling it into information that people can use. We can't afford your vaporware hopes and dreams because we haven't venture capital to rely upon. So while Wikia might be able to flit around giving soundbites to gullible journalists, we have to build search engines and deal with reality. Please don't waste our time with pious platitudes - provide some answers or at least contribute.
"So far there has been nothing real about this project beyond what is effectively just another wiki about the idea of a search engine"
"Most search engine developers are too busy trying to survive without having to subscribe to some happy-clappy ethos that could very well put them out of business."
It's interesting to compare the vapid gushing about the project from the news media and the big blogs echoing them, versus the decidedly more critical evaluation of various search experts. You can find this stuff in a few search blogs, and in obscure corners if you look really, really, hard. But it doesn't reach more than a very tiny fraction of the audience.
I should hasten to add I'm not in complete agreement with him, from my somewhat cynical perspective. Earlier I got into some trouble for I think pointing out, for example, to:
"... keep in mind that there's a whole network of digital-sharecropping electronic plantations, excuse me, I mean Wikia "community sites", which can be used *both ways* to eventually support Wikia Search - as recruiting material for free workers and data to build the high-quality index, and as a partner market for users of the search engine."
Think of it this way way: Consider that what Wikipedia is to Encyclopedia Britannica, Wikia Search can be to Google. Much less accurate, but far more popular with certain users. And if you don't have to pay people and don't need to spend money for quality control, it's all pure profit.
Uncommon echo: updated "grub" crawler code available, though the source is in very raw form.
He said it, not me: Wikia's Outrageous Exploitation of the Human Race:
Google, Microsoft, AOL, AltaVista, Yahoo! and thousands more have something in common, can you guess what it is? That's right! They all pay for their crawlers, power bill, servers, and everything else! So why does Wales think he gets to exploit billions of computers around the web for no reason other than to make him money?
Because that's the Web 2.0 way: "Community" for the digital-sharecroppers, cash for the A-listers.
"Wikia, Inc., the for-profit company developing the open source search engine Search Wikia, has acquired Grub, a distributed crawler platform, from LookSmart."
Now, let's follow the money:
Specifics of the deal were not revealed, though it is part of a larger advertising deal between Wikia and LookSmart which was announced last week.
Under the deal, LookSmart will provide text and display ads in Wikia's freely hosted wiki communities, and eventually on the Search Wikia site, Wales said. Ads will be sold by Wikia on either a cost-per-click (CPC) or cost-per-thousand impressions (CPM) model. Inventory not sold by Wikia will be back-filled by ads from LookSmart's distributed ad network.
So ... the ad-provider will give some old searching technology (I downloaded the Linux version, and it seemed to be from December 2002). The legions of free citizen-workers-for-no-money will immediately improve this, again, donating their skilled labor. The company will run ads on the system the free workers build. That's called democracy?
Bonus Link: Beware The Online Collective - Jaron Lanier
The Web 2.0 notion is that an entrepreneur comes up with some scheme that attracts huge numbers of people to participate in an activity online - like the video sharing on YouTube, for instance. Then you can "monetize" at an astronomical level by offering a way to bring ads or online purchasing to people in your gigantic crowd of participants. What is amazing about this idea is that the people are the value - and they also pay for the value they provide instead of being paid for it. For instance, when you buy something that is advertized, part of the price goes to the ads - but in the new online world, you yourself were the bait for the ad you saw. The whole cycle is remarkably efficient and concentrates giant fortunes faster than any other business scheme in history.
[Original! Not an echo. Even a micro-"scoop" :-).]
I can report that the "Wikipedia-model" search project (though run by "Wikia", a for-profit company which is legally entirely distinct from the nonprofit foundation which owns Wikipedia), the much-hyped potential Google-killer drawing much attention from Wikipedia's most publicized founder, now, at last, has some machines for it.
They were even willing to accept my general request for an account. (i.e. just as an interested person, no free labor promised - and note this is not a blogger-bribe, since accounts don't denote special treatment).
Now to try to figure out if there's any likely path where I wouldn't be be taken advantage of, ("citizen-lunchmeat") due to being overmatched and underpowered. Remember, "predations are conversations".
FastCompany has a long article on the Wikipedia-Model Search Engine. Which, as far as I know, is still vaporware. "Wikia"'s CEO (the startup commercializing Wikipedia's influence, though technically not Wikipedia itself), said the following not me:
The community has responded quite enthusiastically so far, catching even Wales by surprise. "I just thought we'll put a couple of developers on it and kind of play with it on the side and see what comes up," he says. "But now there's a huge developer community that's really interested." As Gil Penchina, Wales's handpicked CEO to lead Wikia, says, "Since the news leaked out, people have been lining up, saying, 'I'll clean the toilet bowl, let me in here.'"
As I've said, there's programmers who would pay to get a shot at being part of a Google-killer, so there's going to be no problem in finding free labor. Even fairly skilled programmer labor. In fact, tongue-in-cheek, I'm starting to wonder if they could even finance the company that way.
Given the way Wales and Wikia can arrange the risk factors (i.e, the programmers work for free, Wikia gets the money of any success), they don't have to revolutionize search in order to ultimately get a nice payday for themselves.
In a discovery worthy of echoing, a nugget of information has been found regarding some of the strategic thinking behind the so-far vaporware Wikipedia-model search engine. And wonder of wonders, miracles of miracles, there seems to be a method in the madness. In a New Scientist interview with J-Wales, he states (my emphasis below)
Why are you developing a search engine?
Transparency is what I'm really after, the idea that we can go in and see exactly how web pages are being ranked. We need to have a public debate about it. We just don't know if there is any dishonesty or strange incentives in today's algorithms that rank searches. Since news of this venture broke (see search.wikia.com) we have been contacted by more than one second-tier company that develops search engines. They recognize that acting individually they are going to have a hard time catching up with Google, because Google has so much money and so many great people.
What's your plan for search?
It's too early for specifics, but one thing that has worked is an alliance in which people contribute to a free software project. We saw this succeed with Apache, the open-source webserver. Apache was a tiny group of volunteers, yet the vast majority of its code has come from companies who paid people to work on it. It's essentially an industrial consortium that has been able to fend off Microsoft's closed-source webserver. So it makes sense for second-tier search companies who are falling behind Google to contribute to a free search software project that will make us equal to Google in terms of search quality.
Note, just in passing, he's wrong: We do know that there are "strange incentives in today's algorithms that rank searches". The poster-child there is Wikipedia! (given its search-ranking dominance).
The goal for the project is to get 5 percent of the search market, Gil Penchina, chief executive officer of Wikia, said Thursday in an interview. He doesn't know when the service will be released.
"We're really trying to build a movement to make search free and open and transparent," Penchina said. "We have some servers up, and people are hacking away."
I'm not saying he's wrong, but I have no idea what he means here in terms of existing servers and development. I don't know of any real public information on this project in the last month. It's sort of ironic to be talking about make search free and open and transparent, while being totally opaque.
Then again, I hardly qualify as a member of the Movement, why should I know anything that would contradict a Movement Council Member?
[1, 2, 3, 4, we will go and work some more ... 2, 4, 6, 8, free labor's really great ...]
Segueing to search: Wikipedia founder says to challenge Google, Yahoo
TOKYO (Reuters) - The online collaboration responsible for Wikipedia plans to build a search engine to rival those of Google Inc. and Yahoo Inc., the founder of the popular Internet encyclopaedia said on Thursday.
Wikia Inc., the commercial counterpart to the non-profit Wikipedia, is aiming to take as much as 5 percent of the lucrative Internet search market ...
I've been on the project's public mailing list, and there hasn't been much activity lately. Down in the land of developers outside of Wikia, there still isn't even a machine set up for initial exploration. Not that this means much, as projects go. But in case anyone wants, or cares, to read something besides gush, the citizen-mushrooms are still in the dark here.
I wonder if one Wikia "exit strategy" under consideration is to be bought by Yahoo or Microsoft. Note the search project might not actually produce much in terms of search innovation, but if it prompts an acquisition somehow, that's a big win for the Wikia investors. Again, just speculation on my part.