A typical thesaurus construction project for a scholarly publisher, policy clearinghouse, medical institution, or any other client with a technical vocabulary involves the input of a number of stakeholders.
At a certain point—usually about three-quarters of the way through the construction of the taxonomy—it’s vital to get the input of subject matter experts (or: subject-matter experts, SMEs, or domain experts). These SMEs generally work for the client—often as technical editors, but just as frequently in other capacities—although we at Access Innovations are occasionally asked to provide them.
Figure 1: An SME in his native habitat
In either case, the SME’s job is to review the emergent thesaurus from the perspective of an expert in the field. Even the best taxonomists can garner a finite amount of knowledge through research, especially in complex, complicated, or highly sophisticated domains; without a PhD in particle physics, your branch on quantum field theory will only be so comprehensive.
The SME is, ideally, well acquainted with the state of research in the field, conversant with the current hot topics, steeped in the journal literature, and familiar with both the stable and fluctuating terminologies used by other practitioners in their discipline.
A vocabulary that’s missing important concepts or terms covering exploding areas of research will not present a good public-facing website for your client’s organization. Imagine browsing a computer science taxonomy that’s missing the term “Big Data” in 2014—would you assume the parent of that vocabulary is competent and up-to-date? Therefore, the SME is vitally important, as the end users who will be searching for content in, for example, a large repository of physics papers will very likely be similar to the SME engaged in the review.
Dealing with SMEs can be an excellent and productive experience; unsurprisingly, this process also involves a number of challenges.
(1) [Most] SMEs are not taxonomists: you have to explain it to them.
Your physicist, oncologist, or social scientist SME (unless they happen also to have attended library school) is probably not intimately familiar with tagging, information retrieval, or taxonomies—at least, not nearly to the extent that they understand their chosen area of study.
Don’t assume that whoever wrangled the SMEs for a call with you explained the project, what’s expected of them, or anything else. Be prepared to give a very short expository talk on why the taxonomy matters, what it will do, and why they’ve been asked to participate.
Figure 2: (Probably?) not a taxonomist
The bright side is that SMEs tend to be intelligent, so they’ll pick it up. Just don’t expect to be able to dive right into the hierarchy without explaining the background of the project, the purpose of their input, and what you expect them to do.
This last point is especially important. Explain clearly what you do and don’t expect the SMEs to provide feedback on. You definitely want:
(a) Input on missing terms/concepts. What needs to be added?
(b) Input on NPTs. What other names, especially acronyms, can supplement the existing terms? What is the fancy new name for a term that everyone’s using since Dr. Johnson wrote his famous 2009 paper?
(c) Input on term placement and hierarchical organization. Do the branches make sense? Are the top- and second-level terms a good outline of the field? (More on this below.)
(2) Some SMEs will be more engaged than others.
Almost predictably, some of the SMEs on a given project will take the minimum amount of time to provide feedback (there may even be some initial grumbling), and you’ll never hear from them again. That’s okay; accept the feedback (asking questions if necessary) and let them go back to their job. For many SMEs, their involvement is an extra assignment over and above their normal workload, so it’s understandable.
Invariably, though, you’ll get at least one SME who gets it—who is excited and engaged and thinks the thesaurus is cool and wants to help. This is exactly what you’re looking for, so make sure to match their level of engagement and enthusiasm. Once they get used to thinking about the taxonomy, they’ll be an invaluable resource.
(3) Disagreements between SMEs and taxonomists
See (1), above. SMEs are not used to thinking like taxonomists, so their ideas about term placement, term formation, and warrant are probably not influenced by things like the ISO and ANSI/NISO standards governing thesaurus construction. They will also not be very sensitive to ambiguous terms, and may be familiar only with the portion of relevant content covering their particular sub-area of expertise. You’ll want to watch out for a few specific issues:
(a) Literary warrant. SMEs will want to add terms covering their entire field, not just the terms required to index the content in question. When considering terms suggested by SMEs, remember to check the content for warrant; reject any terms that don’t meet your criteria.
(b) Term placement. SMEs will have ideas that make plenty of sense to them, but violate (for example) the all-some rule. Stand your ground here; no matter how you cut it, “dog food” is not a dog. Be ready to suggest using associative relationships (RTs), and explain why they’re helpful.
(c) Top- and second-level terms structure. This requires a little more flexibility on the part of the taxonomist; while “Particle physics” is clearly a child of “Microphysics” (as it’s a sub-discipline of that field), if your physicist SME insists that it’s a major enough topic to be a Top Term, you should listen.
(4) Spec creep: how much time for SME review?
A taxonomy, as we know, is a living document that’s subject to constant revision and review, but at a certain point you have to call it complete and deliver the project. This is where your enthusiastic SME can cause problems; they will want to make tweak after tweak ad infinitum. Set a schedule for SME reviews, including a timetable for providing material, getting feedback, integrating that feedback, and returning the revised taxonomy. One more round of changes is acceptable, but if you allow for more, it’ll never end.
Try to allow for about eight hours to process the feedback from each SME. Each suggestion, addition, term move, and deletion needs to be considered carefully, so make sure to allow your taxonomists time to properly weigh SME input.
(5) How many SMEs do you need?
This really depends on the size—and, moreover, scope—of the vocabulary. A thesaurus covering All of Science will require more reviewers than one on Acoustics. If you have many SMEs, try to keep them from stepping on one another’s toes.
(6) Tips on presenting the taxonomy and soliciting feedback
Ideally, you can provide a hierarchical display (naturally, a read-only version) that the SMEs can access; this allows them to see the entire term record, including non-preferred terms (NPTs), related terms (RTs), and multiple broader terms (BTs).
In conjunction with a hierarchical view (if possible), the best mechanism for SME feedback is [still] a spreadsheet with a hierarchical display of terms. (If you can, provide each SME with just the branches of the hierarchy that they’re being asked to review.) A spreadsheet allows the SME to make comments, changes, suggestions, additions, and other input using colors, adding cells, or leaving remarks in adjacent fields. Make sure that the taxonomist can see the feedback at a glance, so they don’t have to spend time poring over the document looking for comments.
(7) You don’t have to integrate every single comment–you’re the filter.
On receiving SME feedback, the taxonomist’s job is not to make every change suggested by the SMEs; rather, the SME’s input is raw material for the taxonomist to consider using. In other words, the SME’s expert opinion has to be run through the taxonomist’s filter to accept, reject, or re-format for inclusion in the taxonomy.
Figure 3: A grain (or more) of salt
On the other hand, have respect for the SME’s expertise. Be flexible when you can, and try to accommodate the SME’s point of view wherever possible. Oftentimes the SME will, for example, make a suggestion to add a term that already exists phrased another way (a conceptual duplicate); this can trigger the addition of an NPT, changing the preferred version of the term, or some other action—one that was not intended, but nevertheless turned out to be useful.
Figure 3: Best. Present. Ever.
SMEs can be a great gift for any taxonomy project—if you have a strategy, provide a clear set of expectations, and maintain good communication throughout the process.
Bob Kasenchak, Project Coordinator
Access Innovations
