Patient-reported outcomes (i.e., questionnaires or scales) are increasingly being used in ophthalmology to evaluate the effectiveness of treatments, including cataract surgery.1-4 While several visual function questionnaires have been developed for this purpose, some of these have, however, been used in other disease groups as well.5-7 For example, the Visual Function-14 questionnaire (VF-14) was developed to measure functional limitations caused by cataract and the outcomes of cataract surgery,7 but has been shown to be valid in retinal diseases and glaucoma too.8,9 Another questionnaire, the Impact of Vision Impairment questionnaire (IVI), was developed to assess the participation in daily activities by visually impaired individuals.7,9-11 Recently, using Rasch analysis, three subscales of the IVI (accessing information, mobility and independence and emotional well being) were shown to be valid in a cataract population too.10 This adaptability suggests that such questionnaires could potentially be administered to populations other than the one they were validated for, provided the validity of these questionnaires has been demonstrated in these other populations also. The excellent performance of the IVI in a cataract population encouraged us to assess in this population the performance of another questionnaire specifically developed for elderly people with vision loss: the Adaptation to Age-related Vision Loss Scale (AVL).11

The AVL scale was developed to meet an identified need in the field of vision rehabilitation: to assess the psychosocial adaptation of elderly people to vision loss. Strong psychometric properties have been reported for the AVL scale.11 A shortened version of the AVL scale (i.e., AVL 12) has also been proposed recently with sound psychometric properties.12 Although the reliability and validity of the AVL scale had been previously demonstrated, this was carried out using traditional classical test theory (CTT).15,16 A major criticism of this method is that the scoring (and, thus, the interpretation) is based on ordinal-level data.1,13,14 Modern psychometric models, specifically Rasch analysis, offer several advantages over CTT, including the important property of invariance: this means that the item parameters and person estimates do not depend either on the items administered or on the people to whom the questionnaire is administered.18-20 Currently, Rasch analysis is being used to develop new measures21-23 and to evaluate existing ones (for example, IVI24 and VF-1415) to determine whether or not the assumptions of the Rasch model are met. The AVL scale has not yet been examined using Rasch analysis. While the AVL scale was developed to assess the adaptation to vision loss resulting from chronic ocular conditions such as age-related macular degeneration (ARMD), other conditions such as cataract cause vision loss, albeit temporarily until surgery. Depending on the country or on the particular region within a given country, the length of the waiting list is variable and patients may have to wait for a considerable amount of time (average waiting period = 3-4 months at our centre) before undergoing cataract surgery. In the intervening period, therefore, patients would have to learn to adapt to their vision loss (which is not as severe in magnitude as with ARMD). In this context, we wanted to investigate if adapting to vision loss was a cause of concern in our cataract population. Therefore, the aim of the present study was to establish whether or not the AVL scale fits the stringent requirements of the Rasch model for the measurement of the adaptation to vision loss in patients with cataract.

The use of the ‘don’t know’ category ranged between 9% (item No. 1) and 55% (item No. 2) among the participants.

The person separation was poor, indicating that the AVL scale was unable to distinguish between desired strata (i.e., at least three) of participant ability (see Table 2). One item (item No. 2) was a misfit, violating the first requirement for unidimensionality. This was further supported by PCA of residuals, which, with an eigenvalue of 2.4 for the first contrast, indicated multidimensionality. The pattern of positive and negative item loadings in the first contrast defined two dimensions, with the five positively worded items comprising half of the items in the “positive dimension”. In an attempt to restore unidimensionality, it was essential that the second dimension, consisting of these five items, was removed and, as a result, these items were deleted. The only way to maintain items from a second dimension is to treat them as a separate scale. Upon testing, these items did not form a separate valid scale (person separation = 0.0). The deletion of the five items left 19 items for further analysis. The items were also poorly targeted to this population. A mean person ability of 1.56 logits indicates that many of the participants did not endorse any of the items as representing issues for them.

Although unidimensionality was restored through the deletion of the above-mentioned items, the person separation continued to remain below acceptable levels (1.24). Again, the previously misfitting item (No. 2, infit MnSq, 1.51) was still a misfit, as was an additional item, (No. 19, infit MnSq 1.32). This misfit indicated that they were not in tandem with the rest of the items in contributing to the measurement of underlying trait. So these items were deleted, one at a time, starting with the most misfitting item; i.e., item No. 2. Thus, 17 items remained. However, there was yet another item that misfited (item No. 8, infit MnSq 1.34), which required deletion. Finally, 16 items remained that fited the Rasch model; i.e., that exhibited infit MnSq values within the desired range (0.7-1.3). Person separation, however, failed to improve (1.22) (see Table 2). This indicates that further revision was not going to result in a valid instrument.

The AVL was developed to measure adjustment to latelife or age-related vision loss in elderly people. Considering cataract is one of the leading causes of vision loss worldwide, we hypothesized that the AVL scale would be suitable for the assessment of adaptation to vision loss in an elderly population currently awaiting cataract extraction. However, the results from the present study indicate that the AVL scale failed to meet the expectations of the Rasch model in this cataract population.

Two fundamental problems surround the use of the AVL scale in this cataract population. First, there was poor person separation, which indicated that the scale was unable to discriminate among the various levels of adaptation to vision loss. This may partly be related to the use of the dichotomous rating scale. It has also been suggested that dichotomous rating scales are disliked by older patients because they are too restrictive.29 A polytomous rating scale is likely to provide greater information about the construct hierarchy, as has already been achieved for the shortened version, the AVL12.12 An added reason for the poor discriminating abilities of the AVL scale could be related to the lack of variation in the levels of adaptation among the participants, with a majority of our participants suffering from only mild visual impairment (mean binocular visual acuity = 6/9.5-1). The content of the items in the AVL scale reveals that they appear to be better suited to the severely visually impaired population. Thus it is likely that the AVL scale may function differently (perhaps better) in another cataract population that is more severely impaired, for instance, in regions such as Africa where the threshold for cataract surgery is lower than that in Australia.30

In comparison with chronic conditions that result in visual impairment, such as age-related macular degeneration, ocular disorders such as cataract cause only temporary vision loss, reversible through surgery. Recent advances in surgical techniques coupled with heightened awareness regarding the benefits of cataract surgery mean that cataract patients can expect surgery to restore their vision to preoperative levels.31 Thus, these patients often have no long-term concerns regarding their vision loss. Such a belief may have resulted in an extreme scoring pattern, leading to a large ceiling effect. The average participant had higher logit score (mean = 1.54, see Table 2) as he/she disagreed with most of the items, indicating a better adaptation to vision loss. This could mean that these participants with cataract didn’t have significant visualrelated problems. However, all cataract patients were drawn from the cataract-surgery waiting list, and the key indicator for cataract surgery is, precisely, visual disability attributed to cataract.32-34 Therefore, this population, by definition, does have visual disability, a fact that was also previously demonstrated.35

A lack of unidimensionality was a second limitation of the AVL scale in the present study. Unidimensionality is a fundamental requirement of the Rasch model. Indeed, recent observations that even slight multidimensionality can affect person estimates have further emphasized the importance of unidimensionality as a measurement property of a questionnaire.36 A lack of unidimensionality was being caused by the five items that were positively coded in the AVL scale, indicating that they did not appear to measure the same underlying construct as the rest of the items. However, the actual content of these items does not appear to be drastically different from the rest of the items and offers no immediate suitable explanation for their multidimensional behavior. Nevertheless, it is likely that some participants may have got used to disagreeing with most of the negatively worded items in the first half of the questionnaire (i.e., items 1-18) and that, as a result, they would have misread the positively worded items (i.e., items 19-24) in the latter half, resulting in unexpected responses. While the authors of the AVL scale coded the five items differently from the rest of the items to deter an acquiescence response set, it may have proved to be counter-productive. The most misfitting item was item No. 2, related to services for the visually impaired. Items misfited for a variety of reasons, such as poor construction, and therefore they run the risk of being poorly understood or they may also be ambiguously worded. However, a more likely explanation in the present case would be related to missing responses to this item from a little over half of the participants (55%), indicating that a majority of the cataract patients may have been unaware of the services that exist for the visually impaired. In the original development of the AVL scale, as well as in its subsequent shortened version, the authors however, recommended the AVL scale to be best used as a unidimensional measure. Factor analytic techniques were used to examine unidimensionality during the original development of the AVL scale. Technically, these analyses require interval-scaled data, and not the ordinal data derived from questionnaires.11 Nevertheless, Rasch analysis can also make use of ordinal data, but the important difference is that it converts ordinal into interval-level data, which are used in the PCA of residuals. Consequently, Rasch analysis provides a sound approach to test the unidimensionality of a set of items that yield ordinal data.17 Although unidimensionality could finally be restored once the multidimensionality-causing items (i.e., the positively coded 5 items) were deleted from the AVL scale, the discriminating abilities (person separation) of this AVL scale failed to improve.

The results from the present study perhaps indicate that a questionnaire should only be used for the specific condition(s) for which the questionnaire has been developed and tested. However, as pointed out in the Introduction, questionnaires such as the VF-14 and the IVI, while originally developed for the cataract and for the visually impaired population, respectively, have also been shown to be valid in other disease groups.8-10 This is because both these questionnaires contain items related to visual disability or activity limitation, a characteristic common across ocular conditions, be it cataract or glaucoma. Similar to the IVI, the AVL was developed for the visually impaired population, but the contents vary significantly. The IVI, in addition to the activity limitation items, also contains items related to emotional well-being. This scale has been shown to be valid in the cataract population.10 In comparison, items in the AVL scale refer only to the adaptation to vision loss, an attribute that is more applicable to people with severe visual impairment and, therefore, which is not applicable to disorders such as cataract that result in rather mild vision loss, as it happens in our population.

In conclusion, the AVL scale is not a suitable instrument for the assessment of adaptation to vision loss in an Australian cataract population. Thus, as envisaged by the authors of the AVL scale themselves, the items may better suit older persons whose visual impairment is due to other age-related conditions (such as ARMD) that cause irreversible vision loss.