In the previous tutorials, we took a closer look at how to conduct a systematic search step by step. Let's recap a little bit. We start with Scoping search, where we become familiar with the topic. During that process, we narrow down our research question, and identify the key concepts in our question. Then we identify both controlled vocabulary and free text search terms for the concepts in our question. Then we identify the databases where we will conduct our searches. For each of the databases we choose, we built the appropriate search strategies. If necessary, we use prefabbed search strategies, such as study design filters and topical hedges to get a more focused result set. If possible we also search outside of the published literature to help overcome publication bias. That's all good, but one thing to keep in mind is that the search process is an iterative one. A systematic search is never a one-shot deal. We need to constantly evaluate, validate, or verify our search results, and revise and re-run the searches if necessary. Now that could be because we've made mistakes in our search strategies, but that usually happens when the concepts in our research topic cannot be properly described using existing controlled vocabulary terms, and therefore, not properly indexed in databases, making them harder to find. Whatever the reason for our imperfect initial search is, our goal is still a search as thorough and exhaustive as possible. So we will need to take further steps after all the efforts discussed in the previous tutorials to ensure that we find studies that might have been missed. There are several techniques for validating and verifying search strategies, and catching possible missed studies. First of all, you can examine the reference list of the studies that have already been identified as being relevant. This is especially useful for those studies that came out recently. Research studies usually have a literature review section, and the references of those reviewed studies are usually included in its reference list. So from there, we can get a list of potential relevant studies as identified by the authors. This will give you a high precision rate, because the list is already a filtered list. Not everything in the reference list is guaranteed to be relevant, but there is a high chance that it will be. The opposite of this technique is to check which studies have citied those studies that have already been identified as relevant. This is especially useful for those studies that were released long enough ago to accumulate citations. Again, not all articles that cite a known relevant study are guaranteed to be relevant, too, but you will have a higher precision rate in such a search. Cited reference search, and citing reference search allow you to follow the thread of research development on the topic. Another technique for verifying your search, and catching possible missed studies, is searching for all studies done by specific authors that have already been identified as influential or prolific in the area of interest. Author searching can result in a biased sample of references because individual authors usually represent their own line of argument. But, again, our purpose here is to catch possible missed relevant studies, and validating our search strategies. We don't completely rely on author searching. Just as author searching, hand searching specific publications is another way of finding missed studies and validating search strategies. This is especially useful if studies in the area of interest is more likely to be published or released in a number of key publications. You will know if this is true when you explore your existing search result set. The rational for hand searching is manifold. Your search strategy may not have been good enough to find all the relevant articles in those journals. Those journals may not have been completely or consistently indexed in the databases you searched. As time consuming as it is, hand searching remains an important step in a systematic search for certain research areas. Another necessary step in validating and verifying your search is consultations with experts in the field. They should either be key figures in the field, or have extensive knowledge about the field. They may be able to provide a list of articles for you to start with, even before you run your systematic searches. Because the list has been prescreened for relevancy by an expert, it should have a high precision rate. The list of articles can serve as a basis for your scoping search, or, as we're doing here, they can be used to verify your search strategy. Did your search strategy successfully retrieve all of the known key articles? Are there any outliers? However, just as author searching, the usefulness for consultations with experts is limited to the knowledge, memory, and viewpoint of the experts consulted--so you may end up getting a biased set of results. Also, they're highly unlikely to give you an exhaustive list of studies, so the recall rate will be poor. But, again, we're not solely relying on consultations, we're only using this as either the basis for our scoping search, or as a way to verify our search strategies. So these are some of the techniques for verifying and validating your search: cited reference search, citing reference search, key author search, hand searching key publications, consultations with experts in the field. Now in this process, if you find studies that you missed in your initial search, it then becomes necessary for you to look at the reason why these studies were missed, and consider revising your search strategies accordingly. Questions you could ask yourself are: "Are the missed studies indexed in the databases I've searched?" "If not, should I search a new database that index these articles?" "If yes, why did I miss them? How are the missed articles indexed in the database?" For example, in the Medline database, you could look at what medical subject headings are used to describe these articles--a process known as "MeSH Analysis". MeSH analysis allows you to quickly develop a list of MeSH terms to use for your search. If this is done in the scoping search stage, it'll help you build your search strategy. The technique can also be used after you've identified a critical mass of relevant articles, to verify that no relevant MeSH terms have been missed in the search. After your searches, and several rounds of validation, verification, and revision--you now have a potentially exhaustive set of relevant studies you can review. Before you do that, it is good practice to document your search strategies, and knock out duplicates in the result set retrieved from different databases--and that's what we're going to be covering in the next video. Thanks for watching--see you next time.