Comments: "Wikia Search" interview and state of Wikipedia-model search project

Minor typo... your questions are labeled 1, 1a, 2, and 3, but the next sentence is "Note understanding 1b) requires some context."

Posted by Bennett Haselton at September 27, 2007 02:12 PM

Fixed, thanks.

Posted by Seth Finkelstein at September 27, 2007 02:16 PM

Thanks for highlighting the thread. I don't have time to dig that deep.

So I take it that right now the bot is in use and crawling the world (but bumping against testing limits).

I can see this bot getting banned real fast - if not already - as a story appears on a popular site and a lot of users dump that url into its start search here now sequence.

How many users are they expecting to run this for them?

If the major blog hosters block it, it will be of no use for searching blogs.

I can just see yahoo and microsoft allowing it run over their sites as well.

So without major news sites, ms knowledge base, blogs, what is left that will be useful?

Sites like cryptome would probably just ban it (or worse) because of the abuse they already get from bots.

How much useful content is left?

Maybe a lot and maybe there is a niche for search engine that search everything except major sites and "highly linked"/authoritave pages.

My contribution to the search eco-system lately has been trying hakia and giving them some feedback - just because I like to see competition in the markets.

I don't think wikiasearch will be it.

Posted by tqft at September 29, 2007 06:11 PM

Yes, the bot's in use - but how useful it is, is a different question!

Posted by Seth Finkelstein at September 30, 2007 02:57 PM