Developing Search Filters
The development of search filters by Flinders Filters and CareSearch follows an established methodology designed to be objective and to minimise bias, set out in the published papers associated with each filter (listed here).
The following broad elements are commonly included:
Establishment of an expert advisory group to provide clinical and/or research expertise in such areas as: choice of a gold standard set of representative references; determination of the scope and function of the search filter; advice on key terminology and key articles; advice on publication and dissemination of the search filter.
Creation of a gold standard set (GSS) of references (contained in the database for which the filter is designed) that represent the breadth of information required to be retrieved by the filter. Each reference is considered relevant to the search topic.
Division of the GSS into 3 randomised subsets: Term Identification, Filter Development and Filter Validation. Importantly, the terms are derived from one subset, and development and testing performed in different subsets.
Frequency analysis of terms in the references to identify candidate search terms (both natural language text words and controlled index terms for the specific database).
Iterative testing of candidate terms, singly and in combination, to determine effectiveness of recalling citations in the subsets and entire GSS, and to develop a draft strategy of the best performing combination of terms. The recall performance in the filter validation set gives a sensitivity rating for the search filter.
A post hoc relevance test performed on a set of references (retrieved from the open database) by pairs of reviewers, who are external to the project team and have expertise in the subject of the search filter. This gives a precision rating for the search filter.
Where the search filter has been developed in the Medline database, we create a translation to a version for the PubMed database. This consists of a component that is equivalent to the Medline search, retrieving items from the indexed section of PubMed, together with a component that is created to search the non-indexed subset of PubMed references. This methodology is outlined in the paper by Damarell, Tieman and Sladek. Creating a PubMed version of the search filter allows us to embed the search filter into a URL to provide the searcher with access to a highly performing literature search simply by clicking a link.