Caleb Tucker-Raymond wrote
a nice post yesterday on the
L-net staff information blog responding to a just published article by
Sarah Houghton and
Aaron Schmidt in the July/August issue of
Online. The article (which is not available for free on the magazine site but can be found in some EBSCO databases, Factiva, ProQuest's ABI/Inform Global, and Gale's Business & Company Resource Center) compares web-based chat software to instant messaging software and declares IM the winner.
As someone who has selected both of the software products we've used here at Baruch College (HumanClick, now known as
LivePerson, and
24/7 Reference), I must admit that I've been frequently disappointed by the technical problems these web-based chat products (especially now that firewalls and pop-up blockers, which often cause problems with the chat software, are widespread on the computers of our users). I would actually like to try out IM in conjunction with our web-based chat service as a way of giving our users more options. But I think I should mention here why I think we should stick with 24/7 Reference for now.
The college where I work as a librarian is a busy commuter school of 16,000 undergraduate and graduate students. Most of our students work part-time or full-time jobs; finding time to get to the library to ask for help is often a challenge for our students. Launching a chat reference service in March 2001 was an obvious solution to this problem. Until we joined the 24/7 Reference Academic Cooperative in August 2003 and could be guaranteed round-the-clock coverage of our service, I had little sense of how many students would want help from a librarian at odd hours of the day. Not long after launching the round-the-clock service, I was able to see how many users were logging in for assistance during hours in which the library had never been able to staff its reference desk or its chat service (typically librarians are here in the library to staff the reference desk until 9 PM on weekdays and 5 PM on weekends, although the library itself is open daily from 7 AM to 12 midnight).
Over the past two years, I've been amazed at how many students want to get help in the middle night or on Saturday and Sunday evenings (yes, I do wish we had enough staff here to keep the reference desk open later on weekend evenings). We wouldn't be able to help those folks if we didn't have cooperative coverage of our chat service. Until we can get a viable IM cooperative service up, we're going to stay with the 24/7 Reference software because it has a well-run cooperative.
Users are generally happy with the cooperative service, too. Every time a user completes a chat session, they are presented with the opportunity to fill out a survey (we've only had this survey since we switched from HumanClick to 24/7 Reference in January 2003). Here are some comparisons between survey data before we joined the cooperative (January-August 2003) vs. since we joined the cooperative (August 2003-June 2005):
- "The quality of library staff service in answering this request was...?" Before the coop: 65% said "excellent" and 25% said "good." After we joined the coop: 64% said "excellent" and 22% said "good."
- "Will you use this service again?" Before the coop: 78% said "very likely." After we joined the coop: 77% said "very likely."
- "The ease of using this online reference is...?" Before the coop, 58% said "very easy" and 35% said "easy." After we joined the coop: 64% said "very easy" and 27% said "easy."
Yes, we do get some complaints about the software problems and about librarians that don't know enough about our library's unique resources, services, and policies. Another survey question asks "Were you satisfied with the answer you received to your reference question?" Before the coop, 83% said they were "satisified," 5% said they were somewhat satisfied, and 12% said they were "not satisfied." After joining the coop, those numbers changed: 62% were satisfied, 21% were somewhat satisfied, and 15% were not satisfied. I have to admit that I don't like to see the "satisfaction" question statistics go down, but should also say that I'm not sure if it is an effective question. In looking at the individual survey responses were the users said they were "not satisfied," most of them also responded to the "use the service again" survey question by saying they were "very likely" to use the service again or would "maybe" use it again. My sense is that there is a group of "not satisfied" users who truly did get less-than-stellar service and responded appropriately but there is also another group of "not satisfied" users who weren't pleased to find out from the librarian that helped them that the involved research question they asked might be best handled in person at the reference desk for forwarded to the library's e-mail reference service. As many of us know, students often have unrealistic expectations of how easily they will be able to find information; their frustration is understandable and just part of the learning process.
Someday, when I finish my second master's degree and have the time (as well as the publish-or-perish motivation) to think about writing another article, I hope to do a more detailed and methodologically rigorous analyis of user expectations and experiences with our chat service. I recognize that the statistics I present here are probably all problematic, although they probably are indicative of a general approval of the service (77% say they are likely to use the service again, the statistic that is probably the most telling of overall satistfaction).
I should mention here that when Sarah and Aaron were writing the article, I responded to their request (posted on Dig_Ref, I believe) for thoughts about IM vs. web-based chat software. Unfortunately, I took too long composing my reply to Sarah and got it to her too late to use.
I also want to mention that Aaron
has posted a brief response today about
Caleb's comments about the lack of balance in the article. Aaron notes that"if they would have given us the whole magazine, we would have filled it."
Finally, in the interest of balance, I want to point out something on the new (and highly recommended)
LITA Blog: a recent post written by Leo Klein, who used to work with me here in the library at Baruch College, and is now in Chicago.
In his comments on "top technology trends," he makes his case for IM for chat reference these days, having seen firsthand what it was like here at Baruch when we were using HumanClick and then 24/7 Reference (before we joined the cooperative).
