Cross-Cultural Adaptation and Reliability of a Filipino Dry Eye Screening Questionnaire
Frances Marie DC. Roa-Lingad, MD, Ruben Lim Bon Siong, MD
Dry eye is a common cause of ocular discomfort that compels patients to seek ophthalmologic care. The Tear Film and Ocular Surface Society – Dry Eye Workshop (TFOS-DEWS) recently revisited the definition of dry eye disease (DED) and revised this to encompass the multifactorial nature of the condition and the resulting loss of homeostasis of the tear film.1 Thus, the extensive variability in disease symptoms and etiologies makes it difficult to develop a consistent classification system and measure of disease severity.2
Various questionnaires for dry eye have been in use as screening tools and in measuring disease severity. Validated instruments ensure consistency in evaluation. They can further be used to quantify the impact of the disease on the patient’s quality of life (QOL) and monitor progression.3,4 Two self-administered screening forms are more popularly used: the McMonnies and the Ocular Surface Disease Index (OSDI).5 The McMonnies survey is one of the earliest tools more widely used due to its formalized grading scheme using dichotomous yes or no responses. Validation studies have found this effective as a discriminating tool for dry eye subjects, but not as a measure of symptom or disease severity.4 The OSDI is a 12-item questionnaire with a Likert design; higher scores reflect greater disability (Appendix 1). It evaluates frequency of ocular symptoms, impact on vision-related functions, and association with environmental triggers.5 Studies on the OSDI have shown excellent test-retest reliability and validity as a measuring tool for severity categorization.
Despite extensive research in determining the reliability and validity of health or disease-related questionnaires, suitable cross-cultural adaptations especially in non-English speaking countries are lacking.8 All internationally accepted measures are in the English language and are intended for English-speaking populations.
Following a literature review on cross-cultural adaptations of health-related QOL measures by Guillemin et al in 1993, a set of guidelines for obtaining language-specific questionnaires was encouraged. They recognized that simple translation of measures may not have equitable results due to language and cultural differences. Moreover, the manner in which health problems are expressed may vary with local culture. The proposed guidelines thus offered a methodological approach to overcome the inadequacies of simple translation and produce an equivalent cross-culturally adapted measure. This involved a 5-step process: (1) translation to local language, (2) back-translation to source language, (3) committee review of local and source versions with modifications as necessary, (4) pretesting for equivalence of both versions with bilingual lay people, and (5) re-examination of weighted scores with adaptation of weights of scores to local context as needed.8 The KIDSCREEN quality of life survey of the World Health Organization (WHO) for children and adolescents have adapted this for European translations since the year 2000.9,10 Likewise, Prigol et al in 2011 applied these guidelines for a Portuguese-adapted version of the OSDI on local dry eye patients. They reported good inter- and intraobserver agreement and concluded that the OSDI may be used in Portuguese to evaluate their patients. To our knowledge, there is no published study of a Filipino-adapted version of a dry eye screening questionnaire.
The objectives of this study were to develop a cross-cultural Filipino version of a dry eye questionnaire and ascertain its reliability. Specifically, the OSDI was translated into Filipino using the cross-cultural adaptation guidelines of Guillemin et al and tested on a pool of Filipino patients with dry eye disease. The reliability of the new questionnaire was determined by calculating the coefficient of internal consistency.
A Filipino-adapted version of the OSDI was developed in accordance to the guidelines by Guillemin et al for language-specific questionnaires. The methodological approach is summarized in a flowchart (Appendix 2). The subsequent testing phase was conducted following the approval of the University of the Philippines-Manila Ethics Review Board (UPMREB).
A. Forward Translation (FT)
To produce a high-quality translation that is culturally representative, 2 non-ophthalmologists were asked to independently translate the OSDI into Filipino. The first translator was a writer for a local television company who was fluent in both English and Filipino, with the latter as her mother tongue. The objectives of the study were only partially disclosed to possibly elicit unexpected meanings from the OSDI.8 The second translator was a Filipino cardiologist who was informed of the objectives underlying the source questionnaire and the concepts involved to possibly produce a more reliable version.8 Both parties did not report significant difficulty in the activity. A review committee resolved the 2 outputs into 1 revised forward translation.
B. The Review Committee
A review committee formed by the primary investigator, research adviser, and a lay individual reviewed all translations. The first two are ophthalmologists who see dry eye patients on a regular basis and are well acquainted with the source questionnaire. The third member is a non-medical, adult, female administrative officer who was chosen to represent the target group of patients. This panel produced 1 modified Filipino questionnaire following each forward or back translation process by accepting or editing words or phrases and providing a simple working version that maintained the concept of the questions or items. A decentering technique was employed to render some structure when discrepancies were resolved. This was done by rendering the forward translations, the back translation, and source questionnaire equally important when compared with one another. Thus, the tool was not centered on a particular language, as all were open to modifications. The final forward translation to Filipino was done by the review committee after evaluating all versions.
C. Backward Translation (BT)
Backward translation from the local language to the source has been shown to be a necessary step to improve the quality of the final forward translated version.8 Here, misunderstood ambiguous terms are detected or augmented, and failure to adapt to the cultural target and maintain the original intent of the question can be revealed. The edited FT Filipino questionnaire from the review committee was submitted to the Sentro ng Wikang Filipino, a language institution under the University of the Philippines, for back translation. The back translators were not given a priori knowledge of the content of the source questionnaire to be free of biases while interpreting the Filipino questionnaire and translating it back into English.
D. Final-Forward Translation (FFT)
The review committee compared the BT version and the source questionnaire and further edited the FT instrument to produce a final forward Filipino translation. The FFT was tested on dry eye patients, and the results of which were analyzed for internal consistency.
E. Patient Selection, Pretesting and Testing
The FFT version of the Filipino questionnaire and the original English OSDI were administered by the primary investigator in a form of an interview to new and returning patients of the Dry Eye Clinic who consented to participate. Eligible patients were Filipino adults diagnosed with dry eye disease and deemed capable of answering the questionnaire (i.e. with intact sensorium and orientation to time, place, and person). Individuals were ineligible if they were unable to read, write, and speak conversational Filipino or English, were hard of hearing, and had undergone eye surgery as the corneal sensation may be altered and thus interfere with the symptomatology of dry eye.
Patients were selected via time-based sampling from October to December 2013. In order to determine whether a question or term was understood correctly, the interviewee was encouraged to elucidate his/her understanding in a ‘think aloud process’ as used in cognitive interviews.13-15 The interviewer also used scripted probes to standardize the interviews (i.e. Did you have difficulty understanding this question? What does this question mean to you? Is the question relevant to your condition? How would you have worded this question?). Upon conclusion, the patients were also asked for other comments concerning the questionnaires.
Pretesting: The participants were asked to answer both the original English and then the FFT Filipino questionnaires 5 minutes apart. This was done to gauge similar responses per individual for both languages and to determine the value of a Filipino-adapted dry eye questionnaire. They were first asked to self-grade their proficiency in both English and Filipino from a scale 1-10 at the start of the interview. Highest educational attainment was noted as well. The interviewees were also asked which questionnaire they preferred after both were answered. The comments of participants from the pretest determined if the FFT version needed further revision or may proceed to the testing phase.
Testing: The FFT Filipino questionnaire was then administered to a larger group on subsequent clinic days for reliability testing. The results were used to compute for the overall alpha coefficient of the questionnaire as well as the alpha that would be obtained if each item were removed. The items that produced low alpha values were then omitted and the remaining items were given to a smaller group of patients. Thereafter, the new overall alpha value was computed.
F. Statistical Analysis
The data from the group of patients who answered both the original English and the FFT Filipino questionnaires were analyzed using frequency distribution graphs.
To investigate the internal consistency and precision of a questionnaire, the Cronbach alpha coefficient with a 95% confidence interval (CI) was calculated. This analysis considers the average variance of each item and the average of all covariances between the questions across the current sample of participants. The values fall between 0 to 1 and is expressed as a ratio of 2 variances: the true-score and total-score (error plus true score). Reliability scales of less than 0.6 were considered fair, 0.6-0.8 as moderate, and greater than 0.8 as high. In this study, the alpha of the overall instrument and the alpha that would be obtained if each item were removed were computed. The Filipino questionnaire without the items that would increase the alpha upon deletion was then retested and the corresponding alpha recomputed.
I. DEVELOPMENT OF THE QUESTIONNAIRE
Two similar albeit differently worded forward translations were obtained. The 3 general questions of the original OSDI were: (1) Have you experienced any of the following during the last week? (2) Have problems with your eyes limited you in performing the following during the last week? (3) Have your eyes felt uncomfortable in any of the following situations during the last week? Translator #1 wrote: (1) May naramdaman ka ba sa alin man sa mga sumusunod noong nakaraang linggo? (2) Noong isang linggo, ang mga problema ba sa iyong mga mata ay nilimitahan ka sa paggawa ng mga sumusunod na gawain? (3) Noong nakaraang linggo, ang mga mata mo ba ay hindi naging komportable sa ilalim ng mga sumusunod na sitwasyon? These were differently worded by Translator #2: (1) Naranasan mo na ba ang mga sumusunod sa nakaraang linggo? (2) Mayroon ka bang mga problema sa mata na nakasagabal sa paggawa mo ng mga sumusunod na bagay nuong nakaraang linggo? (3) Nakaranas na ba ng pagkabalisa ang iyong mga mata sa mga sumusunod na pagkakataon nuong nakaraang linggo? The review committee considered the translations to have semantic equivalence to each other and to the source questionnaire, but utilized that of Translator #2 as these were deemed better phrased. In the FT version of the review committee, all sub-questions of question (1) were obtained from Translator #1.
A back translation to English or the source language was done. This was compared to the original OSDI questionnaire. The review committee reassessed the FT based on the BT version and revised the terms as needed to produce the FFT to Filipino (Appendix 3) of the OSDI, with terms that better fit the intended subjects.
A total of 63 patients participated in the study for the pretest and test phases. A summary of the participant demographics is listed in Table 1. There were 59 returning and 4 new patients from the Dry Eye Clinic who voluntarily answered the questionnaires, of which more than 70% were females per category. All participants fell within the ages of 26 to 80 years. All patients underwent a standardized ophthalmologic examination in the clinic apart from taking the questionnaire. Table 1. Participant Demographics (n=63)
Sixteen patients participated in the pretest phase. The highest educational attainment ranged from elementary schooling to college graduate. Most participants (11/16, 68.8%) had an English proficiency self-rating of 5, with the lowest score being 2 (1 participant) and the highest 9 (2 participants). Everyone rated their Filipino proficiency as 10. Figure 1 shows the percentage of similar responses for all items per participant while Figure 2 demonstrates the percentage of similar responses per item in English and Filipino. Most individuals (13/16, 81.3%) chose the same answer in at least half of the items in both languages, though the range of similar responses varied from 41.7% to 91.7%. The item Q9 had the lowest percentage of participants giving the same responses, while Q2 had the highest. It was noted however that for Q2, 50% of the participants had to have the word “gritty” translated to Filipino, while 25% asked for a rewording of the term and understood more the phrase “bit of sand in the eye.” From the other items, Q5 had to be translated to Filipino for 2 participants while Q4 and Q6 each had to be translated once to obtain an answer. Upon conclusion of the interview, 13 out of 16 subjects (81.3%) reported they preferred to answer the Filipino questionnaire, while 1 favored the English version and 2 expressed no preference between both instruments. All patients reported ease and preference in answering the FFT Filipino questionnaire, thus the tool was administered without additional revisions to participants of the next phase.
Figure 1. Graph of the percentage of similar responses per participant who answered both the original English OSDI and the FFT Filipino-adapted questionnaires (n=16).
Figure 2. Graph of the percentage of similar responses per item from participants who answered both the original English OSDI and the FFT Filipino-adapted questionnaires (n=16).
III. TESTING FOR RELIABILITY
Thirty-six dry eye patients answered the FFT Filipino questionnaire in the testing phase. The FFT Filipino questionnaire was fairly consistent with the original English OSDI with an overall alpha coefficient of 0.5958. On further analysis per item, some questions had coefficients greater than 0.6, meaning that the alpha of the whole questionnaire may increase to ‘moderately consistent’ if such questions were removed. From the table, Q4, Q7, and Q8 could be considered as possible candidates for deletion, as they will increase the overall alpha to greater than 0.6. The questionnaire without items Q4, Q7, and Q8 was retested to investigate if the overall alpha would increase. In a small set of 11 patients the overall alpha rose to 0.7576, which is moderately consistent.
Table 2. Reliability of the Filipino dry eye questionnaire using the Cronbach alpha coefficient with: A. all items (n=36) and B. with items Q4, Q7, and Q8 omitted (n=11).
In a pool of 139 subjects, the study of Schiffman et al in 2000 found the OSDI to have a high Cronbach alpha coefficient of 0.92 and an excellent test-retest reliability of 0.82. However, a reliable instrument is one that measures a construct consistently across time, individuals, and situations. Thus, when the tool is given to a population with a different culture and native language, the results may indeed vary.
The present study developed a Filipino-adapted version of the OSDI based on the guidelines of Guillemin et al for the cross-cultural adaptation of health-related quality of life questionnaires. A series of translation procedures were employed, utilizing different qualified translators, and a review committee to improve the quality of the final Filipino version.
The pretest phase reflected a fair level of equivalence between the FFT Filipino and the original English versions, with a rather wide range of similar responses per participant. However, participant 16, who had the highest percentage of similar responses, preferred the Filipino to the English questionnaire. Participant 6, who preferred the English questionnaire, was a resident of Cebu, a province with a different local dialect. Item Q2 with the largest percentage of similar responses actually had to be reworded or translated to Filipino in more than half of the time, and should thus have a lower value. The questionnaires were administered by the principal investigator in the form of an interview, thus an item may have more similar responses with some help from the interviewer. The results nonetheless support the value in obtaining a Filipino-adapted questionnaire for dry eye.
The Cronbach alpha for the FFT Filipino instrument overall was fairly consistent. Further analysis revealed that removing items Q4, Q7, and Q8 would increase the overall coefficient to greater than 0.6. Upon retesting with the 3 items omitted, the value indeed increased to moderately consistent. The removed items were: Q4 – Naranasan mo na ba ang mga sumusunod sa nakaraang linggo: malabong paningin? (Have you experienced any of the following during the last week: blurred vision?), Q7 – Mayroon ka bang mga problema sa mata na nakasagabal sa paggawa mo ng mga sumusunod na bagay nuong nakaraang linggo: pagmamaneho sa gabi? (Have problems with your eyes limited you in performing any of the following during the last week: driving at night?), and Q8 – Mayroon ka bang mga problema sa mata na nakasagabal sa paggawa mo ng mga sumusunod na bagay nuong nakaraang linggo: paggamit ng computer o pagkuha ng pera mula sa Automated Teller Machine (ATM)? (Have problems with your eyes limited you in performing any of the following during the last week: working with a computer or bank machine [ATM]?). During the interview, only 2 patients knew how to drive and most do not use computers or ATMs. When the FFT version was retested with the 3 items omitted, most participants equated mahinang paningin in Q3 with malabong paningin, which is the omitted item Q4. In these items, answers were usually and inconsistently ‘None of the Time’ or ‘Not Applicable’ which do not equate to the same score. The testing was done in a government hospital in the Philippines, which caters to less financially-abled patients. Results may vary if administered to a population familiar with use of the computer, ATM, and driving.
The new Filipino questionnaire may be further tested for validity as a screening tool and as a measure of severity. Techniques to establish validity in a questionnaire include content validity, criterion validity (measure against a gold standard diagnostic procedure), and construct validity (factor analysis) among others. Schiffman et al determined discriminant validity of the OSDI by testing the differences in OSDI scores by disease severity based on physician assessment and through a composite disease severity score created for the study. Their study also included a factor analysis using varimax rotation on the original OSDI and reported 3 subscales within the questionnaire: vision-related function (6 questions), ocular symptoms (3 questions), and environmental triggers (3 questions). These underwent reliability and validity testing as well and were found, along with the overall OSDI score, to be significantly associated with disease severity. As previously mentioned, the alpha of the FFT Filipino questionnaire may also be increased with inclusion participants from private institutions who are likely to use computers and ATMs more often and drive vehicles.
This study was able to develop a Filipino-adapted version of the OSDI. The value of a Filipino questionnaire was reflected in the ease in answering and preference of the new instrument over the original English OSDI. Reliability testing in a tertiary government institution showed the new instrument to have fair internal consistency. The value increased to moderate internal consistency if items Q4, Q7, and Q8 were removed. The new Filipino-adapted OSDI questionnaire may be tested for validity as a screening tool and as a measure of disease severity in subsequent studies.
1. Craig JP et al. TFOS DEWS II Definition and Classification Report. The Ocular Surface. 2017;57:276–283.
2. Feder RS et al. ed. Preferred Practice Patterns of Dry Eye Syndrome: Limited Revision. 2011. American Academy of Ophthalmology.
3. Schiffman RM, Walt JG, Jacobsen G, et al. Utility Assessment among Patients with Dry Eye Disease. Ophthalmology. 2003;110:1412-1419.
4. Gothwal VK, Pesudovs C, Wright TA, et al. McMonnies Questionnaire: Enhancing Screening for Dry Eye Syndromes with Rasch Analysis. Invest Ophthalmol Vis Sci. 2010;51:1401-1407.
5. McGinnigle S, Naroo SA, Eperjesi F. Major Review: Evaluation of Dry Eye. Survey of Ophthalmology. 2012;57:293-316.
6. Dougherty BE, Nichols JJ, Nichols KK. Rasch Analysis of the Ocular Surface Disease Index (OSDI). Invest Ophthalmol Vis Sci. 2011;52:8630-8635.
7. Schiffman RM, Christianson MD, Jacobsen G. Reliability and Validity of the Ocular Surface Disease Index. Arch Ophthalmol. 2000;118:615-621.
8. Guillemin F, Bombardier C, Beaton D. Cross-Cultural Adaptation of Health-Related Quality of Life Measures: Literature Review and Proposed Guidelines. J Clin Epidemiol. 1993;46:1417-1432.
9. Guedes DP, Guedes JERP. Translation, Cross-cultural Adaptation and Psychometric Properties of the KIDSCREEN-52 for the Brazilian Population. Review of Paul Pediatrics. 2011;29:364-71.
10. Tzavara C, Tzonou A, Zervas I, et al. Reliability and Validity of the KIDSCREEN-52 Health-Related Quality of Life Questionnaire in a Greek Adolescent Population. Ann Gen Psychiatry 2012;11:3.
11. Prigol AM, Tenório MB, Matschinske R, et al. Translation and Validation of Ocular Surface Disease Index to Portuguese. Arq Bras Oftalmol. 2012;75:24-8 (Full article in Portuguese).
12. Rubio CJD and Agulto MB. Translation and Validation of a Filipino Glaucoma Quality of Life Questionnaire. Philipp J Ophthalmol. 2011;36:61-68.
13. Hughes KA, DeMaio TJ. Does This Question Work? Evaluating Cognitive Interview Results Using Respondent Debriefing Questions. American Association for Public Research 2002: Strengthening Our Community – Section on Survey Research Methods. Available online at: http://www.amstat.org/sections/srms/Proceedings/y2002/Files/JSM2002-000622.pdf. (accessed September 2013).
14. Hughes KA. Comparing Pretesting Methods: Cognitive Interviews, Respondent Debriefing, and Behavior Coding. 2003. Available online at: http://www.census.gov/srd/papers/pdf/rsm2004-02.pdf (accessed September 2013)
15. Teschendorf B. The Role of Cognitive Debriefing and Linguistic Validation in Instrument Development and Modification. 2011. Avalable online at: http://www.crfhealth.com/sites/default/files/epro-articles/Cognitive%20Debriefing%20White%20Paper%20FINAL%20Jan%204%202011.pdf (accessed September 2013)
16. Fuentes-Páez G, Herreras JM, Cordero Y, et al. Lack of Concordance between Dry Eye Syndrome Questionnaires and Diagnostic Tests. Sociedad Española de Oftalmología. 2011;86:3-7. (Article in Portuguese)
17. Lin IC, Gupta PK, Boehlke CS, et al. Documentation of Conformance to Preferred Practice Patterns in Caring for Patients With Dry Eye. Arch phthalmol. 2010;128:619-623.
18. Miller KL, Walt JG, Mink DR, et al. Minimal Clinically Important Difference for the Ocular Surface Disease Index. Arch Ophthalmol. 2010;128:94-101.
19. Johnson ME, Murphy PJ. Measurement of Ocular Surface Irritation on a Linear Interval Scale with the Ocular Comfort Index. Invest Ophthalmol Vis Sci. 2007;48:4451-4458.
20. Stevenson W, Chauhan SK, Dana R. Dry Eye Disease: An Immune-Mediated Ocular Surface Disorder. Arch Ophthalmol. 2012;130:90-100.
21. Gulati A, Sullivan R, Buring JE et al. Validation and Repeatability of a Short Questionnaire for Dry Eye Syndrome. Am J Ophthalmol. 2006;142:125-131.
22. Dogru M, Stern ME, Smith JA, et, al. Changing Trends in the Definition and Diagnosis of Dry Eyes. Am J Ophthalmol (serial online). September 2005. Available from American Academy of Ophthalmology website. Accessed June 24, 2013.