We are grateful to Griffiths & co-authors for their interest in our systematic review.1 The review was intended to stimulate debate and encourage further research in this controversial area. In the opening paragraph of the paper we note that a review in this field is challenging because of the lack of large randomised controlled trials (RCTs) and that a similar situation in a different field led Joyce et al. to adopt a pragmatic approach to a systematic review2 and that we decided to follow a similar approach.

Griffiths et al. refer to “the CASP checklist”, but in fact there are eight CASP critical appraisal tools for different types of study.3 These include CASP checklists for randomised controlled trials, case control trials, and cohort studies and so were well suited to our task of reviewing a heterogeneous literature.

We now turn to the specific issue that Griffiths et al. raise concerning the CASP checklist for RCTs. This checklist includes 11 main questions, most of which have sub-questions giving a total of 23 items. To include a table with columns for each of these items, in addition to columns identifying the study authors and study design, would not have been practical. The three questions posed by Griffiths et al. were considered in our review. The question “Can the results be applied in your context?” was considered in Column 3 (“Population appropriate?”) of Table 2 and Table 3. The question “Were all clinically important outcomes considered?” was included in Column 6 (“Outcomes appropriate?”) in both tables. The question “Were the benefits worth the harms and costs” can be taken generically in this subject because the benefits from coloured filters in visual stress are simply reduced symptoms and/or an improvement in reading fluency. This is stated in the last paragraph of the Discussion, which notes the safe nature of the intervention. Nonetheless, the penultimate paragraph of the Discussion raises possible harms or costs, explicitly considering psychological effects of coloured filters and possible costs to the patient and family in terms of expense, time, and raised expectations. We did not use exactly the same wording as CASP but instead used wording appropriate to the topic under review.

Griffiths et al.’s claim that the different headings of Table 2 and Table 3 represent “cherry-picking” is incorrect. The caption to Table 2 clearly states that Table 2 excluded “two CASP criteria: masking (it is not possible to double mask overlay studies) and it is assumed that groups were treated equally as all studies are repeated measures trials”. In other respects the domains listed in both tables are the same. Contrary to Griffiths et al.’s assertion, different studies were not assessed according to different criteria, but rather the over-riding limitation of lack of masking in overlay studies was highlighted in the heading to the relevant table. This is highlighted again in the relevant section of the Results where we say “It is not possible to mask participants in a trial comparing coloured overlays with a control (e.g., grey), and this represents a risk of bias across studies”. The issue of controlling for the placebo effect is returned to once again in the Discussion. In Table 3 the column entitled “Masked?” does identify studies that were double-masked, single-masked (none), uncertain, or with no masking. We therefore reject Griffiths et al.’s claim that we developed our own “hybrid RoB rating scale”.

Clearly, in a review by two authors it was not possible to refer disagreements to a third party. Our approach in such cases was to discuss disagreements to reach consensus and this was achieved in every case. The only exception to this was sections of the review relating to the work of one of the review authors in which case the views of the other reviewer took precedence.

Griffiths et al. cite their own review,4 which they describe as more wide-ranging. In our opinion this review does not consider what is perhaps the most important risk of bias: whether the studies under review investigate the target condition. Their review includes studies of quite small populations of people solely selected as having dyslexia or reading difficulties. Yet, coloured filters are not considered as a treatment for dyslexia, but rather for visual stress, and most people with dyslexia do not have visual stress.1 We consider that this, and other limitations of that review,5,6 mean that it was destined from the outset to support a negative stance.

Griffiths et al. ask how our review concludes that the balance of evidence suggests that coloured filters can alleviate symptoms or improve performance in people who suffer from visual stress. There are three main sections of our Results. The first reviews 10 trials with Intuitive Overlays which all found statistically significant improvements in performance with individually selected overlays. In the review and again now we caution about risk of bias in this result because it is not possible to mask participants in trials with overlays. In the second section we review three papers that each describes large programmes of research in which several studies combine to address many of the limitations of overlay studies. We think that the findings of these studies add some weight to the hypothesis that overlays improve performance for reasons that are not solely attributable to placebo and Hawthorne effects. The third section reviews trials using the Intuitive Colorimeter and Precision Tinted Lenses and all studies found significant improvement of symptoms or performance from colour prescribed with the Intuitive Colorimeter. Our review highlights limitations of these studies and we agree with Griffiths et al. that there is more to a systematic review than just counting up the number of positive studies. This is why, despite the large number of studies with positive results, we used modest language in concluding that “the balance of evidence suggests” that coloured filters are helpful in individuals with visual stress.

We are puzzled as to why the review of Griffiths and co-authors makes no mention of the fact that over 80% of studies in their Table 2 using the “Intuitive system” found statistically significant positive results. We accept that such a count is not proof of effectiveness, but neither is it irrelevant. At the least, such widespread positive results are consistent with the fact that these interventions remain popular and the limitations of this research explain why the interventions remain controversial and this issue unresolved.

We accept the very constructive point that Griffiths et al. make in their last paragraph and we agree that there are several areas on which we agree and these are more important than those on which we differ. We would also highlight some additional areas on which we believe that there is agreement in our position and that of Griffiths et al. Before coloured lenses are prescribed it is important to rule out conventional optometric causes of a child's symptoms. Only a minority of dyslexic children have visual anomalies and when a visual problem is found the optometrist should not claim to be treating dyslexia. However, we believe that visual problems, including visual stress, can co-occur with dyslexia and in some cases contribute to reading difficulties. We believe that children who struggle with reading should have an eye examination and their eyecare practitioners should ask about symptoms, including words blurring and moving. For the minority of children who have these symptoms the optometrist may be able to alleviate a visual barrier to the child benefiting from the specialist educational interventions that they will be likely to need.