DJALCHEMI

The limitations of search for supporting learning

Search engines have a high profile on the web, and understandably so. The web changes faster than any human attempt to catalogue it. For most people's purposes in answering specific questions, Google works most of the time — which is a recipe for success.

But the kind of success Google has enjoyed, and the way it has become a part of every web user's vocabulary as well as habits, can create a kind of tunnel vision (the kind where the only tool you have is a hammer, and every problem looks like a nail). Search facilities are necessary and important tools for rich learning resources, and particularly archives, but they have weaknesses, and are by no means sufficient without complementary ways of discovering new material.

As a complement to search, any service that aims to support learning should have some mechanisms for guiding learners (though this doesn't mean constraining them) to explore particular routes.

Shortcomings of Search

Search facilities generally provide no context. The clean, sparse Google home page is a graphic example of this. It provides no hints to users about how they can get the most value from a Google search — how could it, because Google is such a general purpose tool, that no assumptions can be made about users' goals? (By comparison I like the search box on Tom Phillips' site, which says "Search, e.g. cricket, inferno, Marsyas", thus subtly alluding to the eccentric scope of the site contents.)

Reliance on search is too often used as an excuse for lazy navigation design and lack of attention to the semantic value of links. Designers shrug and say, "There's so much content on this site that the chances of users discovering what they want by browsing are modest, so it's better to give them a good search facility and they can rely on that".

Yes, good search facilities are necessary. But there are several reasons why it is not safe to rely on search alone, especially in learning contexts. Firstly, search skills are not evenly spread across all users. User research has shown that "Typical users are very poor at query formulation" and frequently give up if a first search doesn't get them what they want. Secondly, the big selling point for search — that it offers users control and independence in how they discover resources — applies less in learning situations where users may not know exactly what it is they want to find. There are other secondary reasons — see for example this critique of how search engines rewrite the past, obliterate this past in the process, and are unreliable tools for reconstructing the historical record — but I'm going to concentrate on illustrating these two main, related points.

Comparisons of search with alternative ways of 'guiding' learners

A couple of years ago, when I created and curated the Showroom cinema's web links to complement its film seasons, I was asked why this needed to be done and why the cinema audience couldn't just be left to use Google to find out what they wanted. If cinema visitors wanted to find out more about the season of Lukas Moodysson films, they could search for Lukas Moodysson on Google, and the results on the first few pages of would all be at least partially relevant, because there aren't that many Lukas Moodyssons in the world, and most of the web pages about Moodysson's films tend to mention his name. But the cinema also ran a season of recent Brazilian films, and formulating a search for such a relatively diffuse subject area is harder and more likely to generate a mixed bag of search results (should you search on 'Brazil' or 'Brazilian', 'film' or 'cinema', do you use quotation marks to search for a specific phrase, how do you reduce the number of Spanish pages in the results, and so on).

If you just want to find when a particular film was made, or who directed it, Google and specialist databases (e.g. IMDB) can often help find an answer fairly quickly. But search facilities are not so efficient for getting an overview of a topic. Search results frequently contain many resources that overlap or duplicate each other — for example, many reviews of the same film, or several biographical sketches of a film-maker. Part of my job in curating the Showroom web links was to sift through this duplication and identify the best biography, the filmography with the best supporting assets (film stills, credits and trailers), and perhaps two reviews that represented alternative perspectives on a major film. On behalf of the audience I did multiple searches and selected the best results, so that they didn't have to. To moderate my own selections, it would have been useful to get users to rate each link I provided or suggest new links — a later version of the system allows exactly these refinements.

And in my selection of web links I did more than just judge the best resources to answer audience members' likely curiosities. Based on my knowledge of the domain, I tried to anticipate topic areas that the audience might not think to ask about, but might enrich their understanding of the films in the relevant seasons. To take one simple example, for the web links for the Luchino Visconti season, I expected that few of the Showroom's audience would be familiar with the reunification of Italy that took place in the 19th Century... It's not something you'd expect to need to know before going to the cinema, but a basic knowledge of it helps understand two of Visconti's major films, so I included a link to a one page overview of this period of Italian history.

One further example: if you do search Google for details of The Magnetic Fields' 69 Love Songs, the first pages of results give you a few reviews of varying quality, an awful lot of retail sites and 'meta-retail' (offering price comparisons), but relatively little commentary or background context (aside from the official page and some of the better reviews). By comparison the 69 Love Songs information site aims to provide a more guided introduction to the album (of course, my opinions on its usefulness are very biased, since again I created the site). The links to reviews are still there for those who wanted, but selected, sorted and situated in a wider discourse about the album. I hope it's clear that this site is a more useful learning resource than the raw search.

The role of publicly-funded facilities in supporting learning and discovery

But am I loading the dice against search facilities by using Google as the main example? Google is supported by advertising and other commercial revenue, and these considerations arguably 'contaminate' its utility as a learning tool. The idea of a more 'pure', publicly-funded search facility as an alternative to Google seems to have supporters in high places. The Graf review of BBC Online cautiously endorsed the idea of the BBC retaining and developing its search engine, concluding that "There may be genuine public value in the provision of a search function free from [commercial] considerations (i.e. publicly funded)" and "The BBC is one of the most obvious providers of such a search engine...".

Meanwhile this article suggests that the Museums, Libraries and Archives Council is aiming to compete with Google by offering a high quality service of trusted information sources. I asked someone at the MLA about this, and was told that the article rather misrepresents what their Chief Executive has said. Nevertheless the MLA's Knowledge Web will involve them working with providers of public search facilities to recognise and flag learning resources that are seen as worthy of some quality kitemark.

The Knowledge Web document (PDF) is both ambitious in its goals and vague on how they will be achieved, promising "seamlessly [to] link collections distributed across the country, giving easy access by users to millions of items of digitised content", combining "close links with the broadcast media" with the ability for "individuals to create communities of interest where creativity and scholarship can be shared".

The dream of a one-stop cultural resource that offers access to all the nation's treasures from local art galleries to popular TV-style programming is a seductive one for public sector visionaries who like to think big (Ufi trumpeted similarly grand schemes in its early days). But it is their very multi-purpose scope that often defeats them: they create virtual chimeras. The successful online learning and cultural resources like the Internet Archive and Wikipedia are based on very simple but highly scaleable ideas. The online resource that tries to be all things to all people ends up being very far from a 'one-stop' experience.

Conclusion

As I've said before, at the end of this posting, I think it would be a big loss if a resource like the BBC Creative Archive depended on a search facility as its main front-end interface. That might help users whose primary purpose is research or simple indulgence in a particular resource, but would be less helpful for people wanting to bootstrap their understanding of a resource from a low starting point.

Jakob Nielsen describes users' changing approach to search engines in terms of information foraging theory: "the easier it is to track down new resources, the less time users will spend at each resource". Users, he says, are using search to get quick answers to specific questions and then moving on.

While this leaves users in control of their destiny, it does not allow for the kind of reflective activity that usually constitutes part of rich learning experiences. We all know that we have to be learner-centred, but is good learner-centred design about what learners need, more than what they want?