This study aimed to translate the Convergence Insufficiency Symptom Survey (CISS) into the Italian language and assess psychometric properties of the translated questionnaire (CISS_I).
MethodsThe CISS_I was arranged according to guidelines for a comprehensive multistep methodologic process for translating, adapting, and validating psychometric instruments in health care research. The CISS_I questionnaire was administered to 103 volunteers (21.8 ± 2.2 years), students in higher education, at two different times. A complete optometric evaluation was performed including subjective refraction, best corrected visual acuity, near point of convergence, prism fusional ranges to blur, diplopia and recovery, TNO stereo test and prism cover test for measurement of heterophoria.
ResultsThe performance of the CISS_I in terms of validity showed some points of weakness. Sensitivity was 42%, specificity was 74%, positive predictive value was 27% and negative predictive value was 85%. The area under the ROC curve was 0.672. On the contrary, the results showed good internal consistency of the CISS_I (Cronbach's alpha - α=0.89) and good test-retest reliability (ICC = 0.92). Rasch analysis showed good model fit (all items, except one, with infit and outfit mean square between 0.7 and 1.3), good measurement precision (person separation = 2.66) and good targeting –0,81 logits but also some evidence of multidimensionality.
ConclusionsThe CISS_I showed some point of weakness in terms of validity but also good psychometric properties and has been shown to be applicable to an Italian speaking population to quantify the visual discomfort associated with near vision in higher education students. The results show that high CISS_I score is not necessarily linked to convergence insufficiency, while low scores can exclude the presence of this anomaly. The CISS_I can help in interpreting and monitoring convergence insufficiency symptoms in already identified subjects, but it is not suitable for screening a general population of young adults.
Convergence insufficiency is a common binocular vision disorder characterized by an inability to maintain sufficient convergence for comfortable near vision.1 It is a condition in which there is an exophoria at near, orthophoria or low exophoria at distance, a receded near point of convergence (NPC), reduced positive fusional vergence, and a low AC/A ratio.2 Amongst the various non-strabismic binocular vision problems, convergence insufficiency is the most common and has received the most attention.2 The prevalence of convergence insufficiency varies considerably between studies, with estimates ranging between 1 and 33%,3–18 depending on the population studied, the diagnostic criteria (cut-off values used), the association with accommodative dysfunctions and the reason of assessment (screening studies or after seeking professional assessment for a troublesome symptom).19 A high level of comorbidity has been found between convergence insufficiency and accommodative insufficiency.4,20 Convergence insufficiency is often associated with symptoms such as eyestrain and headaches after short periods of reading, visual fatigue, blurred or double vision, sleepiness, difficulty concentrating, and problems with reading comprehension.2,4,21,22 Children with convergence insufficiency present significantly more visual symptoms than children with normal binocular vision,3,23 although not all convergence insufficiency patients present with symptoms.3,13,21 Many of the most common symptoms (such as headache, loss of concentration, re-reading or forgetting recently read text and feeling tired after close work) are not specific to convergence insufficiency.19 The association between convergence insufficiency and symptoms has been investigated by the Convergence Insufficiency Reading Study (CIRS) group24 and later by the Convergence Insufficiency Treatment Trial (CITT) group, which led to the development of the Convergence Insufficiency Symptom Survey (CISS).23 The CISS is a validated questionnaire for quantifying and monitoring symptoms in convergence insufficiency in different age groups21,23 and used to assess the effect of therapy of convergence insufficiency.25 It contains 15 items (questions) and each question requires an indication about the frequency of symptoms on a 5-step Likert scales - from never to always. The score ranges between zero, when symptoms are totally absent, to 60 when all the symptoms are always present. Good discrimination was obtained using a cut-off score of ≥16 for children, and ≥21 for adults.21 The CISS addresses the most common issues regarding near vision problems and has been subjected to numerous studies to determine its validity, reproducibility, and effectiveness in identifying and monitoring the treatment evaluation in individuals with convergence insufficiency.21,23,26 While the CISS was never designed as a screening tool, it is readily available online, possesses considerable face validity, and has proven to have added value in the diagnosis of other binocular disorders with symptoms similar to those of convergence insufficiency, particularly accommodative insufficiency.20 It has also contributed to the quantification of symptoms of visual discomfort in students,27 and the identification of individuals susceptible to visual complaints associated with visual discomfort from 3D displays.28 The availability of CISS in other languages could be useful, but the process of translation and validity has to follow a certain procedure to maintain the validity of the original instrument.29 A translated questionnaire needs to be framed in a new cultural and linguistic context30,31 and it has to be revalidated in order to guarantee the equivalence to the original.32 In a multilingual region such as Europe, translations are crucial to compare the results of different studies and to allow multicenter studies in different countries. CISS was previously translated and validated to Portuguese26 and Spanish.33 This study aimed to translate the CISS questionnaire into the Italian language and assess its psychometric properties.
MethodTranslation procedureAn Italian version of the CISS (CISS_I) was arranged according to recommendations and guidelines for a comprehensive multistep methodologic process for translating, adapting, and validating psychometric instruments in health care research.29,31,34 The processes involved 3 steps:
- I.
Forward translation: Two native Italian speakers, familiar to questionnaires and vision sciences, translated the CISS from English to Italian. The translators were required to emphasize conceptual rather than literal equivalence with the original version of the CISS. A consensus preliminary initial translated version was obtained by the 2 translators.
- II.
Backward translation: the forward translation was given to a bilingual British-Italian vision sciences researcher, who translated the first Italian translation of the CISS back to English.
- III.
Consensual version development: the backward translation was reviewed by 2 native English-speaking researchers in the field of vision sciences of the Optometry and Vision Sciences Research Group in Aston University, who checked the translation for conceptual equivalence with the original English version of the CISS.
This research was reviewed by an independent ethical review board and conforms with the principles and applicable guidelines for the protection of human subjects in biomedical research. The study received ethical approval from University of Roma TRE (3rd December 2019) and performed in agreement with the tenets of the Declaration of Helsinki. All participants provided informed consent after receiving an explanation of the nature of the study.
Participants, aged between 18 and 30 years were recruited as volunteers amongst the students at Roma TRE University, Florence University and the IRSOO Institute (Vinci, Florence) according to the inclusion and exclusion criteria presented in Table 1.
Inclusion and exclusion criteria for subjects enrolled in the study.
The CISS_I (Appendix 1) was administered to each participant before the optometric assessment. To calculate test–retest reliability, further questionnaire responses were requested 15 ± 3 days after the first compilation of the CISS_I. Fifteen days were considered long enough to minimise memory effects from the first completion and short enough that any significant fluctuation in the condition was unlikely. For each question, the possible answers were provided and scored; never (0), infrequently (1) sometimes (2), often (3), and always (4). The CISS_I score was then determined by summing the points for all 15 items, which could range from 0 to 60.
Optometric assessmentThe battery of optometric tests included slit-lamp assessment to detect any ocular anomaly, non-cycloplegic subjective refraction at distance carried out monocularly by a phoropter procedure, best corrected visual acuity (BCVA), near point of convergence (break and recovery), prism fusional ranges to blur, break and recovery, TNO stereopsis test and prism cover test for the measurement of heterophoria. High-contrast monocular BCVA was measured as the logarithm of the minimum angle of resolution (logMAR) at 5 meters using Sloan letters displayed on an LCD optotype system (Vision Chart, CSO, Florence, Italy) at high-contrast. The near point of convergence test was measured by slowly moving an accommodative target (vertically aligned single letters of about 0.18 logMAR, i.e. 20/30, at 40 cm), at approximately 1-2 cm/sec, towards the eyes until the subject reported stable diplopia, or the examiner noted a break in fusion. The measure was repeated three times and averaged. Phoria was measured by cover test at distance and at near (40 cm) with objective prism neutralization.
Data analysesPerformance of the CISS_I instrument, in terms of validity (the extent to which an instrument measures the underlying concept it is supposed to measure), reliability (the consistency of the instrument in measuring the same construct over different administrations), and psychometric properties (such as dimensionality, targeting and Item fit statistics) was explored in several ways.
Predictive validity (whether the instrument can make accurate predictions of future outcomes) was determined by calculating sensitivity, specificity, negative and positive predictive values for the CISS_I to detect convergence insufficiency. Participants were classified having convergence insufficiency when they presented with the following three clinical signs: exophoria at near at least 4D greater than far,35 failure of Sheard's criteria36 or minimum normal positive fusional vergence (PFV) (break <15D),37–39 and a receded (≥6 cm) near point of convergence (NPC).40,41
Criterion validity, traditionally defined as the extent of correlation between an instrument with an existing “standard” or accepted measure which measures the same thing,42 was obtained by receiver operating characteristic (ROC) curve analysis The area under the ROC curve was used to determine the overall ability of the CISS_I score to discriminate between those with convergence insufficiency and those with normal binocular vision. ROC curves are constructed by determining the sensitivity and specificity at all possible scores cut points of the convergence insufficiency symptom survey. Sensitivity values are plotted against 1-specificity values to form the curve. Area estimates between 0.8 and 0.9 indicate excellent discrimination while areas greater than 0.9 indicate outstanding discrimination.
Convergent validity (the amount of correlation with a related measure) was determined by measuring the correlation between CISS_I score with the amplitude of clinical tests, which are taken as part of the diagnosis of convergence insufficiency: break and recovery of the near point of convergence, break and recovery of positive fusional vergence at near, exophoria at near and at far distance and difference between them, and AC/A ratio (a correlation coefficient greater than 0.3, it is considered a test of convergent validity.43
Test-retest reliability of the CISS_I total scores was assessed using Intraclass Correlation Coefficient.44 and the 95% limits of agreement.45 Reliability is considered slight, fair, moderate, substantial and excellent if Intraclass Correlation Coefficient is comprised between 0.01 and 0.20, 0.21 and 0.40, 0.41 and 0.60, 0.61 and 0.80, and more than 0.80 espectively.46 A Bland-Altman plot was used to assess the difference in measurements in the two sessions (test-retest) as a function of the mean between them.
Since the sum of the 15 items of the CISS_I was used to assess the patient's overall symptom score, a Cronbach's coefficient alpha was calculated to assess the cohesiveness of these 15 items. This value measures the internal consistency (reliability) of the 15 items in the survey. It is calculated using the inter-item correlations. For comparing groups, coefficient alpha values of 0.7 to 0.8 are regarded as satisfactory.47 That is, values of Cronbach's alpha greater than 0.70 would indicate that the items in the survey are all measuring the same underlying construct and thus would indicate that summing the responses is appropriate.
To further evaluate the psychometric properties of CISS_I, a Rasch analysis was also conducted, which is a form of Item Response Theory (IRT). Rash analysis allowed evaluation targeting, Item fit statistics and dimensionality of the CISS_I instrument. The results of Rasch Analysis were compared with those obtained for the English and Spanish versions of the CISS.33 For parameter estimation, the joint maximum likelihood estimation method was used.48 The fit of the model was estimated by the unweighted (outfit) mean square of standardized residuals (UMS) and the weighted (infit) mean square of standardized residuals (WMS). The Rasch principal component analysis (PCA) of standardized residuals was used to assess dimensionality.
The statistical analyses were performed with SAS (SAS Institute Inc, Cary, NC, USA) version 9.4. Rasch analysis was run by the Jmetrik software.
ResultsA total 103 participants (mean age 21.9 ± 2.1 years; range 18-29 years; 33% male) completed the questionnaire and attended the optometric evaluation. Descriptive statistics for the main optometric variables are shown in Table 2.
Main optometric characteristics of participants recruited; PFV: positive fusional vergence. NPC: near point of convergence.
Nineteen subjects out of 103 (19.5%) were clinically diagnosed with convergence insufficiency according to the classification by clinical signs adopted in this study (see Data analysis section). The CISS_I score, determined before the optometric assessment (N=103), had a normal distribution (skewness values and a kurtosis close to zero, 0.44 and -0.09 respectively) and a mean ± SD of 16.1 ± 8.8.
Fig. 1 shows the CISS scores in a box and whiskers plot format for convergence insufficiency group of participants (N=19), non-convergence insufficiency participants (N=84) and the overall group (N=103) respectively.
Distribution of CISS_I score in convergence insufficiency group of participants (CI), non-convergence insufficiency group of participants (No CI) and in the whole group. Dotted line indicates the cut-off score of ≥21 which showed for adults good discrimination power of the CISS.21
Fig. 2 analytically shows the Convergence Insufficiency Symptom Survey score for every participant (N=103). A black dotted line indicates the cut-off score of ≥21, which showed for adults, a good discriminatory power of the questionnaire.21 The 19 subjects diagnosed with convergence insufficiency are shown on the left side of the graph and highlighted with a black bar. Thirty subjects (29.1%) had a higher CISS score than cut-off. Among the 19 participants who were clinically diagnosed with convergence insufficiency, only eight (42.1%) showed a higher CISS score than the cut-off. It is also clear that many participants not having convergence insufficiency (22 out of 84; 26.2%) showed a higher CISS score than the cut-off (median 26, range 21–41).
Distribution of CISS_I scores for subjects with clinical signs of Convergence Insufficiency (black bars) and subjects with no convergence insufficiency clinical signs (white bars). The cut-off score of ≥21 for adults which showed good discrimination power21 is also shown. CI: Convergence Insufficiency. CISS_I: Italian version of Convergence Insufficiency Symptom Survey.
Sensitivity of the CISS for convergence insufficiency in this sample was 42%, specificity was 74%, positive predictive value was 27% and negative predictive value was 85%. The ROC curve is illustrated in Fig. 3, the discriminatory capacity of the test is the better the closer the area under the corresponding ROC curve approaches unity. The area under the curve was 0.67.
The ROC plot of Convergence Insufficiency Symptom Survey score in the detection of clinical insufficiency using the CISS_I score. The diagonal line represents the situation in which the test has no clinical significance as it does not allow to discriminate between subjects with convergence insufficiency and those without.
In terms of convergent validity, the CISS_I score showed significant correlation only with the break and the recovery of the near point of convergence, Spearman'rho 0.30 (p=0.002) and 0.33 (p=0.001) respectively.
As far as concerns the test-retest reliability, of the overall number of 103 interviewees, 96 (93.20%) returned their second questionnaire. The mean Convergence Insufficiency Symptom Survey score of these 96 patients was (mean ± SD) 15.8 ± 8.7 (range 1-41) for the first response and 15.3 ± 8.9 (range 1-44) for the retest. The Intraclass Correlation Coefficient was 0.92 (95% CI: 0.88,-0.95). In Fig. 4 is reported the Bland Altman graph that shows that the correlation between the average of the two measures and the difference between test and retest score is not significant. The mean difference between CISS_I test and retest was 0.5 with a SD of 4.7 (95% LoA: 8.8-9.8).
Bland-Altman plot of the differences between the Convergence Insufficiency Symptom Survey scores achieve in the test and re-test against the mean of the two scores. Limits of Agreement are calculated as mean difference ± 1.96 SD of differences, CI at 95%. The Bland-Altman plot indicates a good agreement between the first and second measurement with no bias induced by the CISS_I score amplitude (mean CISS_I test and retest) on the difference between the test and re-test.
Internal consistency of the CISS measured by Cronbach alpha coefficient for the 15-item was good with a value of 0,89.
Rasch analysisItem fit statistics and item measures (difficulty, in logits) for the CISS_I are provided in Table 3. All items, with the exception of one, showed values inside the more stringent criterion (0.7–1.3) proposed by Pesudovs et al.42 and Khadka et al.43
Rasch Fit Statistics and Item Measure for CISS_I. WMS weighted (infit) mean square of standardized residuals. UMS: unweighted (outfit) mean square of standardized residuals.
The Person Separation Index for CISS_I was 2.66, indicating a reliability of 0.88 and meaning that the CISS_I was able to distinguish 3.88 strata of scores. Using the Wright method (a sample-independent method suitable for clinical samples) to determine the number of performance levels across the CISS_I score range, it was found that the CISS_I could distinguish 6.2 levels of symptoms.
In terms of dimensionality, the Principal Components Analysis of the CISS_I revealed an eigenvalue of the first contrast is 2.65 and a raw variance explained <50%. According to Khadka et al.43 this indicates a multidimensionality of the instrument.
The targeting is the extent to which item difficulty matches with the level of participants’ visual abilities. It is the difference between item and person means (difference of >1 logit indicates significant mistargeting). In this analysis the targeting value was –0,81 logits (< 1) suggesting that item difficulty matches with the level of participants’ visual abilities (Fig. 5). In Table 4, the principle Rash analysis results have been compared with the ones achieved for English and Spanish Version.33
The availability of a standardized questionnaires to assess near visual discomfort in students is important since these individuals are likely to experience eye discomfort and vision problems due to the increasing use of digital screens for extended periods. Furthermore, the availability of those in different languages and the demonstrable consistency across countries is therefore crucial. In the present study the CISS was translated and validated in Italian. The performance of the CISS_I in terms of validity showed some points of weakness.
Despite the fact, that CISS_I resulted easily applicable to an Italian speaking population to quantify the visual discomfort associated with near vision, its property to fully identify actual positive cases of convergence insufficiency (predictive validity) appears quite low. Within a generic population of Italian students in higher education the sensitivity was 42%. This means that, among those who have clinical signs of convergence insufficiency, 58% show no significant symptoms. A poor sensitivity of the questionnaire was seen by other authors using the English version.19 Such a low sensitivity indicates that CISS cannot be used for screening in a general population of young adults, because it would return too many false negatives. On the other hand, the capacity of the CISS to identify, within a generic sample, subjects who do not have convergence insufficiency (specificity) is 74%. This means that about quarter of those who have no signs are identified as having a convergence insufficiency. Analyzing the predictive values of the questionnaire, it is possible to see that the probability that a positive test subject is a real positive (positive predictive value), hence the classification of the subject as "anomalous" is correct, is only 27%. Therefore, if the CISS is positive, approximately only in one case out of four these symptoms are attributable to convergence insufficiency. In other words, symptoms often associated with convergence insufficiency are also quite common in young adults without any clinical signs of poor convergence. On the contrary, the probability that a negative test subject is a true negative (negative predictive value), i.e., that the subject's classification as "normal" is correct, is equal to 85%: this means that, if the CISS is negative, the probability that there is convergence insufficiency is very low. Similar results were observed by Horwood et al19: sensitivity of the CISS for convergence insufficiency in their sample was 37.5%, specificity was 76.8%, positive predictive value was 14.6% and negative predictive value was 92.1%. In this study, as a primary screening criterion, prior to being able to volunteer, the participants were asked to confirm that they considered themselves to ‘have had normal eyesight’ apart from the presence of refractive error, since they wanted to research typical responses in asymptomatic participants.
The present results on the Italian version of the questionnaire (CISS_I) show that it would be necessary to improve the sensitivity and the positive predictive value of the questionnaire to make it valuable. However, as already highlighted by Horwood et al,19 the CISS was not designed as a screening test to be used in the absence of clinical testing, but as a tool to measure the treatment outcomes in clinical trials.21 Moreover, in addition to clinical signs, it is important to include assessments of symptoms to fully characterize the impact of convergence insufficiency on quality of vision and life. The correlation between signs and symptoms (convergent validity) are usually modest: CISS_I score showed significant correlation only with the break and the recovery of the near point of convergence. This indicates that the questionnaire is capturing an additional component of convergence insufficiency that is not captured by the clinical assessment as is the case in tear film assessment, when there is a difference between symptoms revealed by questionnaires and clinical signs evaluated by test and examinations.49 This does not necessarily mean that the measures of CISS or the methods of detecting clinical signs are deficient, but rather that CISS is an additional element of the overall impact of this disease process on affected individuals.
On the other hand, the results of the present study showed very good internal consistency of the CISS_I with a Cronbach's alpha of 0.89, similar to the values published for the English and Portuguese version (0.92-0.962123 and 0.8926 respectively). Also, in terms of test-retest reliability, the Italian version of the CISS showed very excellent results with an Intraclass Correlation Coefficient value of 0.92 close the value of 0.88 reported by González-Pérez et al.33
Finally, the Rasch analysis showed that some psychometric properties of the CISS_I appear to be good (see Table 3) and in line with the original English questionnaire (CISS), and the Spanish version of the questionnaire (CISSVE)33 (Table 4). Specifically, data showed a good Rasch model fit (all items, except one, with infit and outfit mean square between 0.7 and 1.3), good measurement precision (person separation = 2.66 and reliability of 0.88) and goo targeting –0,81 logits (< 1).43 However, CISS_I also showed certain evidence of multidimensionality looking at the level of percent of raw variance explained by the measure and at the eigenvalue of the unexplained variance in the first contrast.43
ConclusionsIn conclusion, the Italian version of the CISS matches the properties of the original English version in being able to quantify the visual discomfort associated with near vision. The present results confirm that high CISS scores are not necessarily linked to the presence of convergence insufficiency, whilst low scores can exclude the presence of this anomaly. The CISS_I can be of help in interpreting and monitoring convergence insufficiency symptoms in already identified subjects, but it is not suitable for screening a generic population of young adults and, therefore, it is a tool to be used alongside the clinical analysis of the convergence insufficiency rather than replace it.
PUNTEGGIO TOTALE________
