The Delphi method is a process composed of several rounds (usually between 3 and 4 rounds) in which the participants answer a series of successive questionnaires and receive feedback on the group's replies. The information is collected and analysed, and a consensus is sought semi-anonymously with controlled feedback throughout the process in order to gather a statistical response from the group. This result is achieved by arriving at a consensus that may be able to provide a solution to a complex problem13 or to the question posed.23,24

The selection of the group of participants was made following a nomination process in 3 steps12,21,24-26 to obtain "useful sample"23 and to ensure that multiple realities of the problem studied were gathered.12,21,24 The first step was the identification of the panels. Second, preparation of lists (one for each panel) with specific names that met the requirements established: the participants, that should be specialists, affected or facilitators,13 were competent and relevant, as well as representative or highly knowledgeable of the professional and academic situation in the field of optics and optometry. They were therefore capable of making insightful contributions, and they also agreed to devote the time necessary to take part in the study.23,24 A minimum of 2 years of professional experience was required for the optometrists experts.24 The third step was to prioritize the nominated participants based on their qualifications.

The aim was to collect information taking into account both the professional point of view and the academic point of view in its different approaches.12  Seven panels of participants related to professional practice and optometry training were formed (Table 1): three panels related to professional practice, including optometrists practicing in visual care centers, optometrists practicing in ophthalmology clinics and hospitals, and professional optometrists who have recently begun to practice the profession (less than 4 years of experience). A panel made up of members of the Spanish College of Optometrists (from four different regions: Castilla y León, Galicia, Andalucía and Basque Country to be representative of all Spain) was included because the opinion of members belonging to institutions independent from the university and the labor market was considered a relevant perspective as regulatory agents of the profession.12 Because this study was focused on improving the Optometry curricula at the University of Valladolid, three panels linked to optometry training at this University were included: one panel of faculty members who were optometrists. A panel of faculty members who were not optometrist was included in order to avoid a greater weight of information from a certain area of training26 and because this was the profile of academics who led the current Optometry curricula. Finally, a panel made up of final-year students of the Optics and Optometry degree was also created, since their knowledge of the current academic reality was considered relevant and they represent a group concerned with the subject matter13,27 and final “costumers” of curricula,12 thus ensuring a plurality of opinions.13

The quality of a Delphi study depends, among other things, on having a sufficient number of representative participants13 that allows for a balance between the diversity of opinions, representativeness of the problem, statistical significance, feasible process of response collection, processing and analyzing the information,23,24 group performance and cost-benefit.13 The research team invited 116 participants to participate (in person or by telephone) until reaching 10–18 participants per panel, as it is in line with the recommendations described in the literature24,28 with the exception of the “Students panel” (all students’ cohort, 22 students, was included) and “Practicing optometrists in visual care centers and hospitals” that including 19 participants in anticipation of a possible drop out along the study.

In this paper, a 3-round Delphi study was used (Fig. 2) .24 The 1st round consisted of asking an open-ended question23,24 that was previously validated by an independent group of 8 professionals from the academic and professional environment.23,24 Each participant had to answer the question “What professional competencies (knowledge, abilities, techniques, skills…) should every recent graduate in Optics and Optometry possess as a primary eye care professional?” on an on-line platform (www.typeform.com) with no answer length limit.24 In this round, a list of items (professional competencies) was identified from the participants’ answers.14 In the 2nd round, an online questionnaire (Google Forms) was used to determine the degree of consensus among the participants on the competencies identified in the 1st round.23The items obtained in the 1st round were reduced so that questionnaire 2 was not excessively long and could be completed in 25–30 min, to minimize the number of participants quitting and to avoid answers without sufficient reflection.13,14,17,24 The item reduction process grouped items identified in 1stround with similar description.23 No one of the suggested items were arbitrarily removed.23 To facilitate statistical analysis, each item was transformed into a 7-level LIKERT scale question (1=not at all important; 7=essential)13,21 and participants rated the importance of each of these competencies in the daily work of an optometrist. For every 4 closed-ended questions, an open-ended question was included so that the participants could make any remarks they considered necessary.21 Based on the participants’ answers, the existence of consensus was calculated for each competency. A 3rd round with a new on-line questionnaire (Google Forms) was planned to include just the competencies upon which there was no consensus in the previous round, so that each participant could assess the importance of each one again using the same LIKERT scale. In each question of the 3rd round survey, the average scores obtained from each panel in the previous round were attached to promote reflection, in addition to maintaining the necessary feedback among the participants. This round measured not only consensus, but also the response stability for the competencies without consensus.

Fig. 2.

Schematic representation of the successive rounds used in the Delphi study.

(0.24MB).

Statistical analysis was performed using SPSS for Windows software (version 23.0; SPSS, Inc., Chicago, IL, USA). The non-parametric data distribution of variables was verified using the Kolmogorov-Smirnov test (P<0.05 indicated that the data were not normally distributed).

The response rate for each round of the study was calculated as the percentage of participants who answered to each questionnaire in relation to the total number initially invited to participate. The 1st round included a qualitative analysis (list of items and categories) and a quantitative analysis (analysis of the frequency of each item identified) of the answers given by the participants to the open-ended question (content analysis method).14In the 2nd and 3rd rounds, based on the participants’ rating of each competency, without weighting the panel’ score, and in order to guarantee the greatest statistical rigor, different statistics of central tendency and dispersion were calculated to ascertain the existence of consensus and stability in the responses (Fig. 3).14,15

Fig. 3.

Description of the indicators established for the existence of consensus on the competencies identified by the participants, as well as the indicators established for the existence of response stability in the 2nd and 3rd rounds.

(0.19MB).

According to the literature consulted, consensus exists when an acceptable degree of convergence (proximity or lower level of dispersion of predetermined values) is reached in the individual estimates. A 7-level LIKERT scale was used to rate the questions, and consensus was considered to exist if each of the following 6 indicators were met: a mean score ≥ 5.6; a standard deviation ≤ 1.4; a coefficient of variation < 20%; a median value > 5; an inter-quartile range ≤ 2; and no statistically significant differences among the groups of participants (p > 0.05 Mann-Whitney) .13,20,21,23 The items upon which no consensus was reached in the 2nd round were included in the 3rd round, and this same analysis was repeated with the new answers. For the items which didn't reach consensus in the 3rd round, the stability of the responses was calculated. Responses were deemed to be stable when no significant variation was found in the participants' opinions between the 2nd and 3rd rounds. To consider that there was stability between the assessments of the 2nd and 3rd rounds, the 9 statistical indicators found in the literature had to be met13,15: variation of the mean ±0.5; relative variation of the mean ±0.1; variation of the standard deviation ±0.7; variation of the coefficient of variation < 10%; variation of the median ±0.5, relative variation of the median ±0.1; variation of the interquartile range ±1.0; relative variation of the interquartile range ±0.20 and the percentage of participants changing their assessment by more than 1 point ≤25%. In items without consensus that showed stability no more Delphi rounds were planned, because this suggests that participants would not modify their opinions.

Of the 116 participants who agreed to participate in the study, 115 (99% response rate) responded in the 1st round. The questionnaires for the 2nd and 3rd rounds were only sent to the participants who responded to the 1st and 2nd rounds respectively. 100 participants responded to the three questionnaires, yielding a final response rate of 86%. The number of participants in each panel and the response rate for each round and panel is detailed in Table 1.

The responses of the 115 participants to Questionnaire 1 identified 105 items relating to the professional competencies of optometrists. These were grouped into 6 different categories. Table 2 highlights the most frequently identified professional competencies from each category.

The 105 items resulting from the 1st round were reduced to 55 professional competencies, which were included in the 2nd round questionnaire. In this second consultation, consensus was reached on 38 competencies (69%) by the different panels of participants (Table 3). In addition, the comments provided on the competency concerning the use of diagnostic and therapeutic drugs (#9) recommended splitting it into two different items in the 3rd round, one on the use of diagnostic drugs and the other on therapeutic drugs (#9A and #9B). This meant that the initial 55 competencies of the quantitative analysis of the 1st round finally became 56.

The 3rd round questionnaire aimed to reassess the 17 competencies upon which no consensus was reached in the 2nd round, including 18 questions (#9A and #9B). After the analysis of the answers of the 3rd round questionnaire, consensus was reached on 5 more competencies (#4, #11, #28, #47 and #52) as shown in Table 3. Therefore, between the 2nd and 3rd rounds, a consensus was achieved on 43 out of the 56 final competencies (77%). However, consensus was not reached for 13 competencies, either because of statistically significant differences between the ratings of the different panels (Table 4) or because the ratings showed a high level of dispersion (Table 3). Response stability was found in 8 (14%) of these competencies (#1, #3, #10, #29, #29, #44, #45, #53 and #55) between the 2nd and 3rd rounds (Table 5). Finally, in the remaining 5 (9%) competencies (#5, #9A, #9B, #14 and #54) no consensus or stability in the responses was found. Given that most of the competencies (43 out of 56) had achieved consensus by the 3rd round and that the majority (8 out of 13) of the competencies without consensus showed stability in the participants' responses, a further 4th round was deemed unlikely to improve these results, and the consultation rounds were brought to an end.

This is the first time a Delphi study has been undertaken to identify and reach a consensus on the professional competencies that an optometrist must acquire in their training during the Optometry degree in Spain. Prior to this study, a team mainly formed by Spanish professors collectively drafted, by consensus and in collaboration with several universities, associations and professional colleges, the White Paper for the degree in Optics and Optometry,4 which was later reflected in the regulation of the official curricula for the training of optometrists (Order CIN 727/2009) .3 However, on that occasion no students, practicing optometrists (the end clients of these professional services) took part in the paper.12 In the present Delphi study, according with the literature recommendations,13,24,28 a large and significant number of participants (grouped in seven panels) related to professional practice, academic and optometry training were formed to ensure a plurality of opinions representative of all the sectors involved in the subject matter6,24 and more thorough results.12,13,24

Participants quitting over the course of a Delphi study is one of the main problems of this type of research. For a Delphi study to be regarded as sufficiently rigorous, a minimum response rate of 70% is recommended.25 In this study, the final response rate was 86% on average: 99% in the first round, 89% in the second round, and 86% in the third round, which is much higher than the minimum required in the literature. This high response rate may be due to the fact that there was personal and direct contact with the participants throughout the study in order to attain a higher level of commitment.23,24

In the 1st round, a total of 105 items were identified, which were grouped into 6 categories. The first four categories (core competencies; competencies in anatomy, physiology, eye pathology and ophthalmology; optics competencies; and competencies in optometry and contactology) include competencies that are very similar to the curriculum in force at the University of Valladolid since 2010 and to current regulations.2,29 However, it was necessary to create two new categories grouping together the professional competencies provided by the participants that are not explicitly taken into consideration in the current curriculum or in the existing regulations.

The first of these two categories, was called “competencies in communication, personal skills and professional conduct”, comprising some of the competencies proposed by the participants, which, despite being included as specific objectives in the degree's curriculum,3,29 are only covered in the specific training modules in some cases. Furthermore, most of these competencies appear as learning objectives of subjects that do not have a defined theoretical content29 such as supervised internships, external internships and the end of degree dissertation, which makes it difficult to guarantee a homogeneous learning process for all students. This contrasts with the recommendations of many international organizations, which insist that a competent healthcare professional should not only have the technical healthcare knowledge befitting the profession, but should also need to have communication skills, interpersonal skills, professional behavior, and solid ethical foundations.30-32 Therefore, these competencies should be acquired during the training process in a straightforward and regulated manner.30-32 Theoretical and practical content on these competencies, as well as their evaluation, should be introduced, thus guaranteeing their acquisition27 by all students, like any other professional competency. These competencies are reflected in the English-speaking world, where the curriculum commonly encompasses specific areas for the teaching and learning of competencies related to professional behavior, communication, professional and ethical conduct and responsibility, in addition to covering legal obligations, collaborative practice and continuing education.5,7,9

The second category that was established was called “business competencies”, to include the participants' proposals concerning knowledge of economics and accounting, management skills and running small and medium-sized enterprises, knowledge and skills in product marketing and merchandising, handling of tariffs and knowledge of real products on the market. These facets are mostly not taken into account in the current curriculum at the University of Valladolid, not even as optional or multi-disciplinary subjects.3,29 The demand for this kind of competencies is quite logical, considering that 70% of optometrists practice their profession as employees and a further 15–17% of them create their own business, which means that 85–87% of optometrists practice in a small or medium-sized enterprise.4 Furthermore, the CIN 727/2009 order marks as a specific objective of the degree the competency “Being able to carry out planning and management activities in a service or small company in the field of Optics-Optometry”, which is closely related to this category of competencies suggested by the participants. The relevance of these competencies is confirmed by several organizations such as the Quality Assurance Agency for Higher Education (QAA) of the United Kingdom, which proposes developing expertise in product sales and business matters in optometry programs.33 The competencies required by the General Optical Council (GOC) for professional registration include a section on the sale of ophthalmic products.9 Additionally, the WCO recommends addressing business models in optometry, marketing and merchandising, taxes and financial management, within the “professional responsibilities” section.32 Finally, the Optometry Examining Board of Canada (OEBC) lays out a field called “practice management” in which sound financial management of business must be proven, as well as having the physical and human resources necessary for a safe practice, managing workflow effectively and complying with local and municipal business laws.5

For the 2nd and 3rd rounds, the 105 competencies identified in the 1st round were reduced to 56 competencies, to aid with the quantitative consultation and with measuring the consensus and the participants’ response stability for each of them. After the 2nd and 3rd rounds, consensus was reached on 43 competencies (77% of the total of 56). Out of these, the competencies with the highest ratings (#8, #19, #21, #37 and #38) can be considered to be the most important in the training of optometrists. They correspond to functions of primary visual care mentioned in the definitions of the optometric profession of the main national and international institutions.1,2,5,7 These results suggest a great deal of agreement between the viewpoint of the profession held by the different participants consulted and the current professional situation.

However, in 13 of the 56 competencies (23%) there was no consensus among the participants' assessments. The lack of consensus is mainly due to statistically significant differences among the assessments of the different panels consulted. These competencies relate to both core and specific areas, which could indicate that participants with different professional activities or educational backgrounds put forward disparate assessments. Thus, the competency on basic scientific knowledge (physics, chemistry, biochemistry and mathematics) (#1) is more highly valued by the panel of faculty members who are not optometrists, precisely those who teach this knowledge. The competency corresponding to knowledge of public health, healthcare management and epidemiology (#3) is rated higher by the panel of members of professional colleges, which are in charge of professional regulation, conveying to political and health authorities that optometrists are health professionals at the service of the population and the healthcare system. The competency on research skills (#5) scored lowest in the panel of optometrists practicing in visual care centers, possibly because they see little link between their daily work and research.

Similarly, the competencies dealing with the use of diagnostic (#9A) and therapeutic (#9B) eye drugs got their lowest scores from the panel of university faculty who are not optometrists, as they have no connection to clinical practice. This likely makes it difficult for them to link the impact of using these kinds of drugs to the quality and effectiveness in performing other competencies on which consensus was found among the panels (funduscopies or retinal examinations, differentiating normal from pathological, referral to other professionals). However, there is consensus among the rest of the panels on both competencies, which are highly valued by optometrists working in clinics and/or hospitals (probably due to their professional experience), as well as by members of professional colleges and university faculty optometrists. The latter two groups have been the most active in seeking recognition and integration of these competencies in professional practice, thus bringing it into line with the model in the English speaking world.5,7,9

In the competencies dealing with optometric care in ophthalmic surgery (#10) and with performing advanced visual examinations (#29), the greatest differences are found between professionals working in ophthalmology clinics and hospitals (with the highest ratings) and those working in visual care centers (with the lowest ratings). These differences are probably due to the former being familiar with this technology and aware of its contribution to their daily work in a hospital environment, while the latter do not usually have access to the technology for advanced visual examinations; they likely don't consider this competency relevant since they don't have the chance to handle this technology or treat patients in this manner. However, this situation may change with the introduction of this technology, which will grant more professionals access to it in the medium or long term.

The competency on the design and implementation of programs for education, prevention, and care of visual health and ergonomics (#45) is highly valued by the panel of professional College's members, which is understandable since this competency is one of the main functions of these organizations. However, optometrists practicing in ophthalmology clinics and hospitals do not consider it as important.

The panel made up of optometrists working in ophthalmology clinics and hospitals gave one of the lowest ratings to the competencies relating to knowledge of economics and accounting, management and running small and medium-sized enterprises (SMEs) (#53), and to knowledge of product marketing and merchandising (#54), in contrast to the ratings given by the rest of the panels of participants. The difference may be due to the fact that this group practices their profession as employees in a public or private hospital environment, in which the marketing of products and services offered does not usually depend directly on them. These two competencies are highly regarded by the panel made up of members of professional Colleges, who are aware of the dual orientation (health and business) of the profession.

Finally, the competency on the management of sensitive information, data protection and confidentiality (#55), is poorly rated by the panel of optometrists in ophthalmology clinics and hospitals, as opposed to the opinion of the panels of professional colleges and university faculty optometrists. This difference may be influenced by the former's work in companies under managers who are in charge of making sure these requirements are met throughout the company, while for the latter two panels it is part of their duties.

Out of the 13 competencies that did not reach consensus, 8 showed stability in the responses received between the 2nd and 3rd rounds (#1, #3, #10, #29, #29, #44, #45, #53 and #55). This suggests that even if more rounds were conducted, the assessments would most likely not change or reach a higher level of consensus, so they were taken as final assessments,13,15 and a further round of consultation was thought unnecessary. Moreover, response stability was not found among the panels of participants for 5 competencies (#5, #9A #9B, #14 and #54), mainly because in 4 of them (#5, #9A, #9B and #14), the participants' opinions changed by more than 25%. This implies that, if further rounds of consultation were carried out, it is possible that their responses would continue to shift. Given that this is a sufficiently small percentage of items (9% of the 56 final competencies), a new round of consultation was not believed to be justified either.

The results obtained in our study could be of interest for future academic debates focused on the improvement of Optometry education in Spain. For example, the current Optometry curriculum in Spain devotes 25% of the credits to the teaching of the "basic sciences" (item #1), but our results suggest that this percentage exceeds that of other relevant items rated with higher importance scores (consensus or non-consensus), such as items #9A or #9B. This example of unbalanced dedication reflects the importance that this faculty profile retained in the design of the curriculum, a situation similar to what happened in other reference models in Optometric education, such as in the UK in the 1990s."

No study is free of limitations. The selection of the panels of participants is important in a Delphi study. In this paper, while the panels linked to professional practice and professional colleges of optometrists were composed of experts from any geographical origin in Spain, panels related to the university were only composed of participants associated to the University of Valladolid (faculty members and students). This is because the main objective of the study was to reach a consensus on a set of competencies that would guide the revision of the curriculum taught at that university. The geographical limitation of some of the panels of participants should have a minor impact on the extrapolation of the results. On the other hand, the number of participants was not the same for all panels; e.g., 11 optometrist faculty members took part, compared to the 22 participants in the student panel. This lack of homogeneity is justified18 because in this case there were no more university faculty in the UVa Degree in Optics and Optometry who met the required qualifications for panel 4. Meanwhile, all final year students of the degree were included in panel 6, anticipating that some participants might drop out over the course of the successive rounds of the study.24 The high response rate and the rigorous statistical analysis mean that this disparity in the number of participants in the different panels has a reduced impact on the competencies identified.