Yahoo Search Engine Spiders Directories From File Paths

I just did an experiment and confirmed that the Yahoo spider will try to search a directory from a file. That is, if it sees a URL like http://example.com/stuff/jump.avi , in addition to retrieving that file, it'll try the URL http://example.com/stuff/ . Though Google won't do that (nor will Microsoft). It's easy to test this yourself if you have a website where you can see server logs. This practice has some significant implications for people who claim that trying truncated URLs is improper behavior and even possibly unauthorized access.

Seth Finkelstein's Infothought blog (Wikipedia, Google, censorware, and an inside view of net-politics) - Syndicate site (subscribe, RSS)