WebSeitz/wikilog
Vertical Search Engine
Whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it, and to institute new Government, laying its foundation on such principles and organizing its powers in such form, as to them shall seem most likely to effect their Safety and Happiness.

(backlinks off) (map off)
(search off)
last edited by BillSeitz on Aug 12, 2008 12:46 am

A which indexes documents identified on multiple public web servers, grabbed by a . Could be for research use, or as a public service. (Could also be a on steroids.)

At we intended to use the spider product for this, but never got around to it. was (in 1999 at least) very expensive for this, if you want to spider more than a dozen hosts.

If you're doing this for a public site, you probably don't want to drive users to a local cache of the ultimate content, since it would be a copyright violation - you want to point them to the original destination. Which limits you to spidering free servers, probably. Also, your search engine has to support some structured data (to associate the ultimate [URL] with the index entry) and the ability to use that in a result listing (so that the points to that field, rather than the local cache copy [URL]). For an , you might be less concerned about this (though legally it's just as much of a copyright issue).

See : | | | | | | | | |


 




Bill Seitz, fluxent at gmail dot com, Weblog