In many countries, optometrists are increasingly viewed as the main providers of primary eye care. For example, in the United Kingdom (UK), optometrists are the main originators of referrals to hospital eye services.1 In the UK, optometrists can only practise if they are in training (under supervision) or are registered with the General Optical Council (GOC).2 After university before GOC registration, a period of pre-registration training is undertaken working as an optometrist under supervision. After GOC registration, some optometrists choose to take additional post-registration training leading to a qualification in independent prescribing, allowing them to prescribe medications where necessary.3,4 Other optometric higher qualifications are available.5

It is therefore desirable to monitor the range and quality of clinical practice provided by optometrists and, in particular, a crucial patient management problem: decision-making concerning referrals. However, measuring clinical practice and determining appropriateness of referrals can be challenging.

Written methods have been used to assess clinical decision-making; e.g. case based multiple choice questions (MCQs), and patient management problems. These methods suffer from validity and reliability issues, particularly scoring and consistency of performance across cases.6 Both MCQs and patient management problems introduce a prompting effect when the participant is asked to choose from a list of options. One aim of our study was to design an online presentation that minimised the prompting effect associated with lists.

Vignettes are designed to simulate medical conditions to evaluate skills and measure clinical practice.7,8 The use of vignettes for measuring clinical practice is a relatively new concept for primary care optometry. Other approaches that have been used to assess clinical practice and the content of optometric consultations include record abstraction and standardised patients (SPs)9,10 (trained actors who attend appointments unannounced). A review of these techniques recommended SPs as the reference or ‘gold’ standard approach, but suggested vignettes could provide useful data, especially if computerised. However, there are limitations to all these methods, including selection bias (only more confident practitioners or those who believe they are practising to a high level may choose to participate) and the influence of the Hawthorne effect (improved behaviour when observed).8,10 Comparison between record abstraction, clinical vignettes and SPs has been conducted for three optometric presentations (symptoms of migraine-type headache, recent difficulty with near vision in a patient ‘at risk’ of glaucoma and recent onset of flashing lights in one eye).11 It was concluded that clinical records tend to under-estimate the actual care provided, while vignette scores over-estimate clinical performance. Vignettes prove particularly useful for comparing different groups of practitioners.7 Studies using vignettes are starting to provide comparative data on diagnostic clinical decisions; e.g. between optometrists and ophthalmologists using images of wet age-related macular degeneration lesions12,13 and measuring the impact of optical coherence tomography in primary care.14,15 In this study, vignettes had the advantage of being a cost-effective way of assessing clinical practice across a variety of settings.

Reducing unwarranted variation in referrals may save resources if unnecessary referrals are avoided and may improve quality of care. Variation in referrals by General Medical Practitioners (GP) has been demonstrated.16 Whilst not specific to referrals, variation has been observed in optometric practice which ‘highlighted substantial differences between different practitioners in the duration and depth of their clinical investigations’17 and that ‘there is no such thing as a ‘standard sight test’.18 While variation in management in glaucoma and diabetic eye care has been found by record abstraction, capturing referral data by record abstraction is challenging.19

The first stage in the planning of the study was to develop a vignette clinical decision-making and referral framework for managing risk and uncertainty in primary care optometry (Fig. 1). This modelling process was informed by a critical review of the variation in GP referral decisions.20 Although an older reference, this review captured factors for variation in GP referrals which can equally be applied to contemporary primary care optometrists. However, whilst there are data on the variation in GP referral quality and rate,21 similar data for optometric referrals are sparse. In designing hypothetical clinical scenarios, several factors needed to be considered. First, the primary binary outcome of referral or no referral. Each option could be modified by secondary factors. For example, a decision not to refer could be mitigated by arranging a sooner than usual follow-up. A decision to refer could require the secondary outcomes of deciding on the urgency and route of referral (e.g., routine or fast-track suspect wet age-related macular degeneration). Additionally, each vignette could be classified as either ‘definite’ or ‘grey’ (uncertain). In a definite case, it would be clear to any reasonably competent optometrist whether the patient should be referred. In a grey case, the decision of whether to refer is less certain and it could be possible to make a reasonable argument for either option.

Fig. 1.

A vignette clinical decision-making and referral framework for managing risk and uncertainty within primary care optometry (referral improvement loop in red).

(0.25MB).

A wide range of practitioner, patient and practice characteristics may be involved when optometrists are contemplating their referral management decisions. Decisions should be influenced by clinical and professional guidance,22 supported by Continuing Education and Training (CET)/ Continuing Professional Development undertaken. Having an accurate or confirmed diagnosis may not be the deciding factor, and the decision to ‘refer’ or ‘not refer’ may depend on how the practitioner is contracted, local services and familiarity with these, how the practitioner manages risk and perceived uncertainty, and how they cope with the time pressures of daily practice. Shared decision-making between patient and optometrist, and the influence of easy and unhindered referral access to Hospital Eye Services were not investigated.

Detailed information on UK context, CET, expert panel scoring, the pilot study, final website design, and the recruitment and communication plan can be found in the Appendix.

All participants completed a questionnaire to capture data on time since qualification, higher qualifications, main workplace (independent, small or large multiple practice) and weekly workload (days worked per week, and total number of eye examinations undertaken per week). The vignettes were first applied to a group of qualified optometrists (QOs). Details of the study were publicised to the profession through various media24 and consenting practitioners were asked to complete ten vignettes (set A, pre-CET) and then another ten vignettes (set B, post-CET). Between the two sets of vignettes was a six-month period during which participants gained CET points from undertaking CET approved events or courses of their choice. Participants were free (as are all UK optometrists) to choose whatever CET they wished, including the option of not undertaking any CET in the six-month period. As the current three-year GOC CET cycle was coming to a conclusion, participants submitted to the researcher a full list of CET undertaken, copied from their online (GOC website) record. The CET undertaken during the six-month period was identified from the full three-year data. The participants’ CET points data over three years was compared with a GOC randomly-selected reference sample of anonymised CET data to investigate the typicality of the study population. Data collection took place between 24th February and 31st December 2015.

The second group of participants was newly-qualified optometrists (NQOs) within six months of registration. Approaches were made to several optical companies. One large company with a workforce of approximately 30 NQOs agreed to participate. The same sets of vignettes were used with a two-day NQO clinical training course as the CET. Data collection took place between 12th October and 19th November 2015.

The third group of participants was pre-registered optometrists (PROs) working in practice under supervision. This pre-registration period normally takes place following university graduation and involves regular assessments prior to taking the final assessment (Objective Structured Clinical Examination (OSCE)). Successful completion of the assessments and OSCE enables the PRO to join the GOC register. The CET for PROs was a six-month period of their pre-registration period. Data collection took place between 20th December 2015 and 31st October 2016.

The CET that took place between the two sets of vignettes differed in the three groups. This was unavoidable, because the acquisition of CET points is not a priority for NQOs (compared with experienced practitioners) and CET for PROs is dictated by the requirements of their pre-registration period.

Shah et al.11 used three vignettes to assess the content of optometric eye examinations and compared data from these with approximately 300 standardised patient visits by three patients matching the vignette scenarios. In the present research, vignettes were used in a different way to assess the impact of optometric CET. Two sets (allowing the use of CET) of ten optometric online vignettes were developed. A novel feature of the design was a controlled presentation which minimised the prompting effect (a limiting factor in the use of vignettes to measure performance).26

With the modest sample sizes studied, there were no significant effects of CET. However, the research does reveal a significant association between number of referrals and time since qualification.

In a recent publication, we showed from an audit of referral data that some NQOs may be responsible for a greater proportion of false-positive referrals than other practitioners.26 It is noteworthy that the PRO group performed slightly better in referral decision-making than the NQO group, but this did not reach statistical significance. It should also be noted that, in actual practice (but not the vignettes), referral decisions by PROs are moderated by the supervisor, an experienced optometrist who has undergone additional training.

Most variation occurred in the ‘grey’ vignettes and not by vignette topic.26 Analyses can be found on the data page in the file labelled ‘Vignette results data final’ at: https://optomscenario.wordpress.com

Three QO choices (outlined by the ellipse in Fig. 3) related to a decision not to refer, but instead monitor a vignette case with early Fuch’s dystrophy which did not require treatment. The authors did not consider these decisions to monitor rather than refer to be unsafe practice, but rather demonstrated confidence to make a management decision supported by experience, higher qualifications and the ability to manage risk.

Cognitive biases can influence clinical decision-making, but further research is required to understand their influence and resulting mitigations in eye care.27 Years in practice does not appear to mitigate cognitive bias.28

The authors suggest that several ‘real world’ factors should be considered when assessing referrals made by optometrists in England. These include pressure to make the correct clinical decision based on a single appointment (often when the patient has not been seen previously), fear of an incorrect decision resulting in civil litigation or a complaint to the GOC, and variation related to the setting. For example, a variation in setting occurs when cataract referral decisions are influenced by locally imposed visual acuity thresholds.29 If this occurs, it would be unfair to criticise the referrer. However, in this study, most of the variation related to over-referral of patients with better visual acuity than typically imposed thresholds.

NQOs could be described as ‘safe beginners’ and may have had insufficient exposure and experience with eye pathology within their training and, therefore, may lack confidence in differentiation of abnormal from normal. It is particularly worrying that NQOs had a slightly higher rate of eye examinations per day than the QO group. It would be reasonable to expect practitioners to perform more rapidly with experience, so the figure for NQOs eye examinations per day could be expected to lie between that of PROs and experienced optometrists. The departure of our data from this anticipated trajectory could be explained by the higher than average level of care exhibited in our experienced practitioner group, and/or could indicate that NQOs are placed under undue pressure to perform at a faster rate than more experienced peers would accept.

Although we found no statistically significant evidence of high eye examination numbers in NQOs contributing to over-referrals, our low sample sizes mean that we cannot exclude this possibility. It is noteworthy that the recently published GOC Standards for Optical Businesses30 require UK employers to ensure that (Standard 3.1.3) ‘operational and commercial pressures do not unreasonably inhibit the exercise of professional judgement’. Additionally, Standard 3.4.1 requires that an optical business ‘supports its staff in making referrals and ensures that they only make referrals when appropriate and clinically justified’. It is currently unclear whether these issues will be monitored and, if so, how. One possibility is an annual survey of employed optometrists and opticians to investigate whether they feel appropriately supported with regard to these issues. Our results provide some support for this idea and indicate that data collection from NQOs should be prioritised.

At the other end of the spectrum of experience from PROs and NQOs are experienced optometrists with an independent prescribing qualification. It is intriguing that Fig. 3 suggests that practitioners with this qualification may refer fewer cases than other practitioners. This might be unsurprising as they should be able to manage more cases themselves, and the additional training associated with this qualification could lead to more accurate and less over-cautious decision-making. Alternatively, fewer referrals could be the result of over-confidence. However, a recent study in acute ophthalmology services in Manchester suggests this is not the case as it found clinical decision-making by experienced and appropriately trained independent prescribing optometrists concordant with consultant ophthalmologists.31 In Scotland, the state supports and funds optometrists to gain this independent prescribing qualification.32 It would be interesting to investigate whether state-funded training is a cost-effective intervention to reduce the influx of new cases to over-stretched ophthalmology units. During the COVID-19 pandemic, community optometrists in many areas of the UK have taken over management of some conditions that would previously have required ophthalmology care. Undoubtedly, a wider range of conditions could be managed by community optometrists with an independent prescribing qualification.

In England, although most ophthalmology new patients originate from optometrists,1 ophthalmology letters are addressed mainly to the GP and replies are only rarely received by the referring optometrist.33 For all optometric referrers, but especially NQOs, the referral learning process requires a feedback loop (Fig. 1 shown in red). Without any identification of outlier decision-making and learning from replies, inappropriate referral behaviour is likely to continue unchecked. This study identified over-referrals in some NQOs, but it seems likely that all forms of inappropriate referral behaviour will thrive in the absence of replies to the referring optometrist. Regular communication between ophthalmologists and optometrists34 is essential for a referral improvement loop.35 It supports on-going management of patients when discharged to primary care and improves the on-going quality and appropriateness of referrals.36

While fears of a GOC investigation or civil litigation may be a factor in over-referral by some optometrists, there is no specific published evidence to support this. However, anecdotal evidence in a GOC report of over-referral by NQOs has been highlighted by employers and by hospital optometrists who receive these referrals.37 Some employer interviews indicate ‘an unfounded fear of the regulator and litigation has been fostered in some newly qualified practitioners, adversely affecting their abilities to make confident clinical decisions’.

Computerised vignettes are a useful tool for comparing referral decisions between groups, especially if the vignette design avoids the prompting effect. Despite the study being underpowered, the results suggest that unwarranted variation in optometric referral decision-making exists and that it is possible to identify under-performance. Strategies to reduce unwarranted variation in referrals require further investigation and use of these vignettes has potential to inform this research. However, recruiting representative groups of participants remains challenging.

The research was conducted in accordance with the Declaration of Helsinki (1975) and approved by research ethics committees of London South Bank University and the Institute of Optometry, London.

The website and vignette software were supported by grants from the Central Optical Fund, and Bexley, Bromley and Greenwich Local Optical Committee. The funding bodies had no say in the design of the study, analysis of results, or writing of this paper. We would like to thank the optometrist and ophthalmologist members of the expert panel for their time and clinical expertise, and all those participants who took part, and the volunteers who provided valuable feedback in the pilot study. We are grateful to the College of Optometrists, the General Optical Council, and the optical company who supported the study of newly-qualified optometrists.