Clinical Translation: How Should Clinicians Proceed in Light of Findings that Alcohol Use Disorder Diagnostic Criteria and Assessment Instruments May Suffer from Important Shortcomings?

Clinical Translation:
How Should Clinicians Proceed in Light of Findings that Alcohol Use Disorder Diagnostic Criteria and Assessment Instruments May Suffer from Important Shortcomings?

Cassandra L. Boness, MA and Kenneth J. Sher, PhD
University of Missouri

Recent research indicates that instruments designed to assess diagnostic criteria for alcohol use disorder (AUD) may be limited in their ability resolve an AUD diagnosis in a reliable and valid manner, which could have unfortunate downstream consequences. For example, our research team has demonstrated that the way AUD symptoms are operationalized, or defined, can greatly affect overall prevalence rates (Boness et al., 2019). This is at least partially the result of the widely varying severities of the various possible operationalizations of AUD symptoms (Lane et al., 2016). Further, there is evidence that the items used to assess a given AUD symptom, such as withdrawal, may be misunderstood by respondents and, thus, result in false positives for that criterion (Boness et al., 2016). In order to most accurately identify individuals with an AUD diagnosis, there is a need to address these known shortcomings.

As a way to address some of the limitations with current AUD diagnostic criteria and operationalizations, a recent brief report published in The Journal of Studies on Alcohol and Drugs entitled, “The Case for Cognitive Interviewing in Survey Item Validation: A Useful Approach for Improving the Measurement and Assessment of Substance Use Disorders” calls for the use cognitive interviewing when developing diagnostic assessments. Cognitive interviewing is a technique that aims to provide a systematic way of identifying and reducing measurement error associated with the structure and content of assessment items. It has the goal of understanding how respondents interpret a given item and select a response, and can be used to provide feedback on the candidate items and response scales, suggest alternative wording, and improve the validity of specific items in their ability to assess the construct of interest.

In our brief report, we argue that cognitive interviewing is especially well suited for reducing measurement error resulting from issues such as varying item operationalizations and inaccurate interpretations of items, particularly during the measure development stage and specifically for AUD diagnostic assessments. This can help improve the validity and reliability of AUD assessment items and resulting diagnoses. Although cognitive interviewing is often utilized as part of the research process, it can also inform clinical practice and assessment. The current discussion focuses specifically on how clinicians can address known issues with AUD criteria and assessment tools using approaches informed by cognitive interviewing.

The major lesson clinicians can take away from our brief report and the cognitive interviewing literature more broadly is that respondents may not always understand items in the way they are intended to be understood. Indeed, in our brief report we demonstrated that assessment items typically intended to assess for alcohol tolerance are understood differently across respondents. When asked, “In the past 12 months, did you find that your usual number of drinks had much less effect on you than it once did?” some participants reported that they were comparing their usual number of drinks now to when they first started drinking. However, others reported that they were comparing their usual number of drinks now to their freshman year of college (i.e., several years earlier), to the beginning of the current year (i.e., a few months before the interview), to one year ago, and to the start of the academic year (i.e., 4–5 months earlier). This is problematic given that respondents were answering on the basis of different time frames.

In an independent study (Boness et al., in preparation), college students participated in cognitive interviews with the goal of improving alcohol-induced blackout assessment items. Several of the blackout items included vague time frames for memory loss or impairment duration such as, “a short period of time,” “several hours,” “large stretch of time,” and “hours at a time.” Participants tended to have widely varying interpretations of such time frames. For example, when participants were asked to define “a small part of the day,” they responded, “a few minutes or so,” “1 to 2 hours,” and “half the night.” Widely varying interpretations may therefore have impacted participants’ responses to the items and resulted in false positives or false negatives. This is especially concerning given the duration of memory loss is commonly used by college students to distinguish between en bloc (complete memory loss) and fragmentary (partial memory loss that can be recalled with cues or reminders) blackouts (Miller et al., 2018).

In order to address issues such as those described in the examples above, clinicians can take several steps. First, when choosing assessment instruments it is important to consider the psychometric properties of the instrument and to take into consideration whether cognitive interviewing, or similar techniques, have been used to evaluate how participants interpret and respond to the items included. This information should be considered in the interpretation of the client’s responses and overall diagnostic impressions. Second, when assessing someone for a possible diagnosis of AUD, it is important to ensure the individual understands the items in the way they were intended. This can be achieved by asking the participant to paraphrase what they think the question is asking them, for example. When there is misunderstanding, the item can be clarified, and the response can be revised as needed. This is fairly easily achieved when conducting clinical interviews and semi-structured diagnostic interviews given their relative flexibility but may require more intentionality for highly structured interviews and questionnaire measures. If clients cannot be directly probed for their understanding of items, clinicians should use their judgment in determining whether or not the response to the item is valid. Third, it is worth considering how a client’s personal attributes, such as age, first language, race/ethnicity, and cultural background, as well as other contextual factors may inform their interpretation of an item and their resulting responses. Finally, clinicians must incorporate any information gained from the aforementioned practices into all diagnostic impressions, assessment reports, and notes, being sure to provide explicit information on how these factors may have influenced a participant’s responses and if/how that changes the clinician’s diagnostic determination.

References

Boness, C. L., Gatten, N., Treece, M., & Miller, M. B. (in preparation). A mixed methods approach to improving the measurement and assessment of alcohol-induced blackout.

Boness, C. L., Lane, S. P., & Sher, K. J. (2019). Not all alcohol use disorder criteria are equally severe: Toward severity grading of individual criteria in college drinkers. Psychology of Addictive Behaviors, 33(1), 35–49. https://doi.org/10.1037/adb0000443

Boness, C., & Sher, K. J. (2020). The Case for Cognitive Interviewing in Survey Item Validation: A Useful Approach for Improving the Measurement and Assessment of Substance Use Disorders. Journal of Studies on Alcohol and Drugs. https://doi.org/10.31219/osf.io/u35m4

Lane, S. P., Steinley, D., & Sher, K. J. (2016). Meta-analysis of DSM alcohol use disorder criteria severities: structural consistency is only ‘skin deep.’ Psychological Medicine, 46(08), 1769–1784. https://doi.org/10.1017/S0033291716000404

Miller, M. B., Merrill, J. E., DiBello, A. M., & Carey, K. B. (2018). Distinctions in alcohol‐induced memory impairment: A mixed methods study of en bloc versus fragmentary blackouts. Alcoholism: Clinical and Experimental Research, 42(10), 2000-2010.

Do you have any recently published work that you wish you would have had more room in the manuscript to discuss clinical implications and applications? For this issue, we would like to focus on any cutting edge work you might have related to the COVID-19 pandemic. We would love for you all to share recently published work in this area and give us more information about how your research findings could be useful for clinicians. Please limit responses to 1,000 words and send to dana.litt@unthsc.edu by February 1, 2021.