Federated search bake-off

For anyone in the market for a federated search tool, it can be hard to compare products. As Sol Lederman at the Federated Search Blog notes, it's a challenge "because a number of vendors don’t have publicly available demo applications" or because, "where the demos do exist, it’s hard to compare them because they’re not searching the same sources."

Lederman is proposing to vendors that they allow him to set up demo site for their products on his blog in which each product is linked to the same set of publicly available databases (e.g., Medline, ERIC, etc.) He's asking his readers to submit a list of the databases that should be included in the standard set of ten that all products will work with. I think Lederman has got a great idea. Whether any of the vendors actually participate is an open question. Regardless, it's worth submitting your list of databases as comments on Lederman's original post.

Our library at Baruch College is about to launch a federated search tool using Serial Solutions' 360 Search. I am on the committee that was charged with figuring out how to set up the product; I've learned a lot of ugly truths about how search results are retrieved via federated search and how they are aggregated. Much of the literature in the library field that that I've read talks about the relationship between federated search and usability or federated search and information literacy; I'd like to see more that gets into the nuts and bolts of how well these tools actually work and where they are typically hamstrung by the database vendors, who often give the federated search vendors less than optimal gateways into their data. Having a central place where you could test drive different federated search tools would not only reveal how different interfaces work, it might also show how the connectors that federated search companies build for you are not all created equal.