In the western world stroke is the third most common cause of death with a reported incidence of >200/100,000 persons annually.1 More than 10 % of these ischaemic strokes are attributable to atherosclerosis of the internal carotid artery.1 Large landmark trials have confirmed the effectiveness of carotid endarterectomy (CEA) for reducing the stroke risk by r emoving the embolic atherosclerotic plaque. In the 1990s, as endovascular techniques became more widespread, carotid artery stenting (CAS) was introduced as a minimally invasive alternative therapy. Initial observational and case studies of CAS suggested that the procedure was feasible, safe and effective in treating carotid stenotic disease with high technical success rates. In 2004, the US Food and Drug Administration (FDA) approved the first endovascular device system for CAS in the US in patients at high risk for CEA.2 In 2011, based on the Carotid Revascularization Endarterectomy versus Stenting Trial (CREST) trial results,3 comparing CAS with CEA for patients with symptomatic and asymptomatic carotid disease, the FDA expanded the indication to all patients with high-grade carotid artery stenosis.4 Although the CREST trial showed that CAS was equivalent to CEA for the primary composite endpoint of peri-procedural death, stroke or myocardial infarction, the trial did not resolve all the controversies surrounding CAS.3 Similar to the results of European randomised controlled trials (RCTs) such as the Stent-Supported Percutaneous Angioplasty of the Carotid Artery versus Endarterectomy (SPACE)5 and Endarterectomy Versus Angioplasty in patients with Severe Symptomatic carotid Stenosis (EVA-3S),6 CAS seemed to be associated with a higher peri-procedural stroke rate (offset by a lower rate of myocardial infarction in the CREST trial). To date, the heterogeneity of the major trials and their different inherent methodological problems make it difficult to present clear guidelines for either CAS or CEA. This fact is highlighted by the disparity in recommendations made by the different vascular, neurological, radiology and cardiology societies in their respective consensus documents across the continents.
What the large CAS trials have also pointed out is the impact of patient-specific and physician-related factors on the outcome after CAS. This analysis has identified certain risk factors; these include patient age,7–10 gender,11 symptom status,7,8 timing and type11 of symptoms before CAS,8–10 patient co-morbidities,9,12,13 concurrent medications,7 and specific anatomic configurations of the arch and carotid vessels.14 Physician-related factors such as level of training and experience of the lead interventionist, as well as the overall hospital volume with the CAS procedure, should also be taken into consideration.15
This article will outline the effect of operator experience on procedural outcome and provide strategies, which may increase performance and patient safety after CAS. Therefore, the focus will be on factors including generic training, pre-procedural rehearsal and the process of optimising patient selection for CAS, but not on risk factor management and CAS-specific (technical) device developments.
The Learning Curve for Carotid Artery Stenting
Learning curves are a well-recognised phenomenon for any new or technically challenging procedure. A clear learning curve for CAS was objectively documented by Lin et al. in 2005.16 In this report 200 patients were divided into four consecutive groups of 50 patients and peri-procedural complications were analysed. The authors observed a significant increase in technical success rate after 50 procedures and a concomitant reduction in total procedural time and contrast volume used. The 30-day stroke and death rate was 8 % after 50 interventions and 2 % after a 100 cases, but continued to decrease significantly after 150 procedures (0 %, p<0.05). It was speculated that the decrease in procedure time (from 60 to 40 minutes) lead to enhanced results by decreasing the embolic risk associated with reduced catheter manipulations and in situ thrombosis of indwelling catheters.
Analysis of the Carotid ACCULINK/ACCUNET Post Approval Trial to Uncover Rare Events (CAPTURE 2) study by Gray et al. confirmed that site volume also correlated strongly with the incidence of major complications (death and stroke).15 This inverse relationship was even clearer for individual physician volume. The conclusion that a patient should be operated in a high-output centre by a physician with a high caseload seems justified. Based on the CAPTURE registry data the minimum number of carotid artery stenting procedures to achieve a major complication rate below the American Heart Association (AHA) guidelines of 3 % (a threshold set for patients undergoing CEA) was 72 patients. This is considerably higher than what has previously been suggested by national and international societies,17–19 and also higher than the threshold used to enroll trial physicians into randomised trials like CREST.3
Nallamothu and colleagues focused their efforts on examining the relationship between operator experience and 30-day CAS mortality rates using national Medicare administrative data from 24,701 procedures performed by 2,339 physicians between 2005 and 2007.20 This provides an insight into the consequences of actual ‘real life’ practice patterns as opposed to the controlled environment of pre-credentialed RCTs. They found a 30-day mortality of nearly 2 % among Medicare beneficiaries. This is significantly higher than mortality rates for elderly patients documented in the major trials and registries (0.7–1.0 %).3 Suboptimal patient selection with inclusion of older patients may be partly responsible, but inexperienced CAS practitioners and low-volume centres also played a significant role. Moreover, the median annual operator volume in Medicare beneficiaries during the study period was only three per year (interquartile range, 1.4–6.5). These low-volume operators (<6 CAS procedures per year) were found to have an increased odds of death compared with patients treated by high-volume operators (>24 procedures per year). Furthermore, a clear learning curve was noted – inexperienced practitioners (cases 1–11) had twice the mortality rate compared with more experienced interventionists (>12 procedures). As a result of the evident learning curve and severe complications associated with CAS, several subspecialty societies have created various credentialing and consensus documents dealing with CAS competency requirements.17 The larger trials comparing CAS to CEA have also instituted credentialing processes to ensure that the physician investigators overcome the initial learning curve of CAS prior to participating in a trial.3,21 The rigorous credentialing process in CREST did indeed appear to lead to superior results compared with the previous European SPACE and EVA-3S data, where inclusion criteria for trial participants were less stringent and controlled.3 The results led on to The CREST Abbott Vascular (Santa Clara, CA) premarket approval supplement presentation on January 2011 at the US FDA Circulatory System Devices Advisory Panel where they presented the Rx Acculink™ carotid stent system for consideration of an expanded indication for use in a standard operative risk population.4 Once again in this presentation a clear relationship between the peri-procedural complication rate and temporal inclusion in CREST became evident, signifying obvious within-trial learning (i.e. increasing physician experience during the trial duration resulted in a risk reduction with time for patients undergoing CAS) (see Figure 1).
Although the trials and registries mentioned above have highlighted that physician and centre experience influence outcome, societies still need to create definitive guidelines for physician credentialing, physician training programmes, and patient and device selection criteria in a bid to enhance the CAS outcomes. The trials looking at the learning curve indicate that the minimum caseload necessary to obtain experience in CAS seems higher than put forward in the initial consensus guidelines.
Strategies to Improve Procedural Performance and Patient Outcome
Virtual Reality Simulation and Physician Training, Proctoring and Credentialing
Although deeply rooted into medical education, the traditional training model devised by Halsted using the ‘see one, do one, teach one' approach has inherent drawbacks and exposes patients to risks associated with the aforementioned learning curve. It is unstructured, lacks objective feedback and may be ethically challenged as it puts patients at unnecessary risks, especially during complex high-risk procedures such as CAS. Nonetheless, this Halstedian approach is still the gold standard in most training institutions, where trainees progressively learn endovascular procedures on patients under experienced supervision. Proctoring is also a fine-tuned example of this type of a training process.
There are several alternatives to learn CAS and to acquire the necessary skills to enhance procedural performance. These include industry-sponsored courses and carotid simulation training modules. Generic training using virtual reality (VR) simulation has several potential advantages over the classic approach aiming to provide skills acquisition. It allows the trainee to learn in an environment where he or she is central to the learning process, as opposed to the stressful patient-centred theatre environment. Skills acquisition can take place at the trainees own pace, taking into account the variability in innate capabilities of each individual learner. Thus, this kind of training shifts the emphasis from caseload learning to proficiency-based learning. This learning process is more structured, allows trainees to effectively learn from mistakes and permits a gradual increase of procedural difficultly within the training process until predefined benchmark criteria are fulfilled. All this is achieved without harm to the patient. The training process does not limit itself to the interventionists, but team training is also possible in a simulated environment. Additionally, crisis scenarios that are not often encountered in real life can be practised until proficiency is reached both by the interventionist and by the team. It is important to stress that learning new procedures entails more than psychomotor skills acquisition alone. Any training curriculum has to incorporate a cognitive component as well, including elements regarding patient evaluation and selection, risk factor management, procedural technique, clinical decision-making and peri-operative patient management.
At present the available courses are not standardised or based on predefined and validated benchmark criteria, partly because it is difficult to define these due to the heterogeneity of the physicians performing CAS (cardiologists, [neuro]radiologists, neurosurgeons and vascular surgeons). Professional organisations representing the major three endovascular specialties have already tried to list specific criteria to steer a credentialing process for individual operators performing CAS.17–19 These criteria include cognitive and technical aspects as well as minimum volume requirements. Once again there is variation across the different subspecialties regarding threshold requirements, but in general, volume requirements seem invariably low (often around 25 cases) especially in light of the recent learning curve data.4 One should also not overestimate the role of absolute volume, as quantity does not necessarily guarantee clinical or qualitative competence (i.e. experience does not always equate to expertise) – ‘slow’ learners may reach the same expertise as ‘quick’ learners, but will need a larger caseload to reach the necessary level of competence.
The FDA has also encouraged educational initiatives, and in particular the incorporation of simulation technology into training packages, prior to granting privileges for physicians wishing to perform the CAS procedure on patients.2 This interest in VR simulation has sparked efforts to scientifically validate this technology for training and assessment purposes. There is growing evidence that VR simulators have a role in physician training and credentialing by shortening and flattening the learning curve. Patel et al. showed that cardiologists performing five consecutive VR carotid arteriograms resulted in a significant improvement in total procedure time, contrast use, fluoroscopy time and number of errors with catheter manipulations.22 Our own European Virtual reality Endovascular RESearch Team (EVEREST) has shown that a two-day CAS course, including supervised simulation training, leads to significantly improved performances with respect to procedure completion time, fluoroscopy use and delivery–retrieval time of the embolic protection device (EPD). Procedural errors observed post-course reduced significantly and expert raters regarded 60 % of the interventionists competent during a non-complex virtual CAS procedure after attending the two-day course compared with 0 % before the course.23 Chaer et al. reported their experience with VR simulation in training residents in non-complex endovascular procedures and for the first time observed transfer of the acquired benefits to the real operative environment leading to higher quality performances on patients (VR to operating room [OR] transfer).24 Endovascular simulator training does require close mentoring including expert feedback to ensure correct and prompt skills acquisition. The simulator training itself cannot be regarded as a standalone teaching tool.25,26
VR simulation seems to be particularly useful for inexperienced interventionists or more experienced practitioners wishing to learn new procedures. Current generation VR simulators probably lack the fidelity to refine the skills of experienced interventionists, potentially because of biomechanical limitations such as impaired haptic feedback. However, as mentioned by Satava, training on a simulator is not about the simulator itself, but about the structured and proficiency-based training curriculum of which the simulator is an integral part.27 This curriculum includes cognitive components, errors identification and technical skills acquisition to predefined expert benchmark levels.27 Although currently expert-derived benchmark levels of performance of skill for CAS are not yet available, their identification will eventually allow us to define the levels of skill that are needed prior to treating real patients. Subsequently, simulators will be able to evaluate performances and may be used as a reliable and objective credentialing tool. Van Herzeele et al. have already shown that high-fidelity simulators are able to objectively differentiate level of CAS experience (i.e. construct validity) across four groups of experienced interventionists based on basic assessment parameters recorded by the VR simulator (procedure time, fluoroscopy time and number of recorded angiograms).28 The simulator derived error scoring is currently not a valid mode of assessment and needs refinement, but expert-based rating scales have been validated to assess the quality of the executed procedure on the simulator in the interim.29 The European board of vascular surgery exam is an example where board certification is granted only after basic endovascular skills have been evaluated while working on the Simulator for Testing and Rating Endovascular SkillS (STRESS) machine. Assessment is carried out using previously validated, expert-derived rating scales for basic endovascular skills, as automated error scoring recorded by the simulators themselves is still unreliable.30 Once benchmark levels of skill are defined and the fidelity of the simulator error scoring is improved, patient safety may be enhanced by objective evaluation and credentialing prior to independent CAS practice. This process of creating proficiency-based simulator curricula for interventional procedures is the focus of continuing research in simulation science.
The benefit of VR simulation training is not solely reserved for the inexperienced practitioners. Experienced CAS practitioners can use VR simulation as a tool to safely integrate new CAS technology that arises during the course of their career. An example is the application of new proximal EPDs that aim to protect the brain from peri-operative (micro) embolisation, especially during crossing of the lesion and angioplasty of the stenosis.31 These include the Mo.Ma® Ultra Device (Medtronic Invatec, Frauenfeld, Switzerland) and Gore® Flow Reversal System (W. L. Gore and Associates, Flagstaff, Arizona, US) that use either stasis or reversal of flow to minimise the embolic burden during the CAS intervention itself. Moreover, the MICHI™ neuroprotection system (Silk Road Medical Inc, Sunnyvale, Califona, US) not only uses flow reversal but also a cervical approach to avoid any manipulation in the arch and has been shown to reduce the embolic load significantly.32 In order to learn the procedural sequencing together with the endovascular skills required to use the Gore flow reversal system safely, an endovascular VR simulator module has been created and may thus minimise the negative consequences of the learning effect of these complex devices on patients.
Virtual Reality Simulation and Procedure Rehearsal
Patient-specific VR rehearsal, also referred to as ‘procedure’ or ‘mission’ rehearsal, is a new technological advancement within simulation science, which allows patient-specific computerised tomography (CT) scans to be incorporated into the simulation software (see Figure 2), enabling rehearsal of actual patient cases. This patient-specific role complements the established role of simulators as a ‘generic’ training tool. The concept of pre-procedural rehearsal has already been implemented successfully in other high-stake industries such as the military and aviation.33,34 Likewise, VR simulators may also be used to detect potential difficulties, surgical errors, and analyse near miss incidents in the medical domain to prevent subsequent complications in patients. Due to its procedural complexity, this technology was initially created for the CAS procedure. Technically, patient-specific rehearsal seems a practical and effective tool to plan CAS cases pre-operatively, evaluate different approaches, identify potential hazards and optimise endovascular tool selection. Patient-specific VR rehearsal seems an ideal tool to complement a tailored approach to each individual CAS case.
Case reports certainly indicate that this technology could be useful in the immediate pre-operative setting.35–38 Initially the process of incorporating individual patient data into simulators required technological support from the simulation company, which proved a time consuming and costly process.35 The introduction of the commercially available PROcedure Rehearsal Studio™ software (Simbionix USA Corp, Cleveland, Ohio, US) was a definite step forward, as it allows physicians to create these patient-specific simulations themselves in a cost-effective manner. These simulations were found to exhibit a high degree of realism although the quality of the patient-specific rehearsal is dependent on the quality of the source CT (or magnetic resonance imaging [MRI]) data.39
One of the most evident benefits of patient-specific VR rehearsal for CAS is the opportunity for physicians to evaluate how specific endovascular tools may interact with the anatomy of an actual patient during catheterisation of the common and internal carotid artery, placement of the EPD, stent and balloon. Research has shown that patient-specific rehearsal can indeed provide interventionists with a detailed evaluation of case complexity and influence both experienced and inexperienced interventionists, most notably for the optimal fluoroscopy C-arm position, choice of selective catheter, choice of sheath or guiding catheter and balloon dilatation strategy.40 The pre-operative knowledge of the optimal endovascular tools could result in using fewer endovascular tools with a decrease in hazardous manipulations in dangerous anatomic regions such as the aortic arch. Furthermore, it could also prove to be cost-effective as the use of unnecessary material is avoided.
Subsequent research established that in a simulated environment, both from a quantitative and qualitative perspective, CAS procedures were performed to a higher standard if a pre-operative VR rehearsal had taken place. The operation was carried out more rapidly, the fluoroscopy pedal was pressed less frequently and for a shorter duration, and fewer errors were committed during the intervention.41 Error reduction was apparent by a decrease in excessive catheter manipulation, a reduction in the suboptimal selection of vascular stents, and a decrease in suboptimal C-arm angles to delineate the different vessels during the procedure. Catheterisation of target carotid vessels was accomplished faster and the EPD was deployed for a shorter duration, reducing the risk for clot formation and stroke in high risk areas such as the internal carotid artery. In addition, it was established that full-length rehearsals are not always necessary and part-task rehearsals, focusing on the most crucial procedure steps, may be as effective while saving time.42
Patient-specific rehearsal may also be used as an ultimate form of pre-operative warm-up. Warm-up exerts its effect by increasing the physical and mental preparedness (cognitive arousal effect) and by increasing the individual’s perceived control of the situation (i.e. confidence).43 VR simulators seem to be an ideal tool for warm-up, using either patient-specific or generic preset simulated cases. Furthermore patient-specific rehearsal can also be viewed as an excellent training tool, applied predominately in the latter stages of the training process, where it may tailor and facilitate the transfer of skills acquired in the laboratory environment to treat real patients in the actual angiosuite in a safe manner (VR to OR transfer).
Another area of interest is patient-selection. VR rehearsal may be able to provide information on procedure feasibility, specific hazards and risk stratification, and aid the physician in his or her decision-making process. This technology can be used in conjunction with existing expert-based anatomic scoring systems for CAS.44 These scoring systems intend to guide less experienced practitioners in patient selection by identifying patients at higher risk of peri-operative complications
Limitations of the current generation of PROcedure rehearsal software relate to the use of ‘static’ Digital Imaging and Communications in Medicine (DICOM) imagery as source data for the simulations and its influence on simulation fidelity. Although CT or MRI data provides information on the anatomy and specific configurations of the arch and carotid vessels, information such as the degree of calcification, vessel wall atheroma or vessel wall thickness are not incorporated into the simulation. Therefore, biomechanical properties including the reaction of vessels and stenoses to rigid guidewires, stents and balloons are not always replicated accurately. Future software updates will have to incorporate these biomechanical properties to improve the simulator fidelity and allow all facets of the real operation to be replicated.
If the fidelity of patient-specific VR rehearsal is further improved, this opens the door for additional applications. VR rehearsals can be used as a post-operative debriefing tool to re-enact unexpected events or complications that occurred during surgery, and to understand factors associated with positive and negative operative outcomes. Using this tool to both rehearse the procedure beforehand and refine operative technique afterwards can be considered an example of ‘deliberate practice’, as described by Ericsson in 2003.45 This kind of targeted practice of new or evolving skills (as opposed to repetitive practice of previously obtained skills) may prevent arrested development and ensure that experienced interventionists continue to learn during the course of their career, eventually resulting in expert performance.
Patient-specific VR rehearsal can also be applied to educate patients and provide them with a detailed plan and prognosis of intended treatment. If increasing fidelity results in VR simulation mimicking every aspect of human interaction, these simulators could even serve as a tool for procedural prototyping – using them as ‘guinea pigs’ to develop and refine new surgical techniques and products within the domain of CAS and other complex endovascular procedures.
Team Training
The initial case reports evaluating procedure rehearsal for CAS primarily focused on its role as a technical adjunct for the interventionist performing the procedure. Although this is crucial, patient-specific rehearsal has the potential to be much more than a technical tool alone, as it can also be used to train the entire interventional team and improve non-technical performances. Non-technical skills such as teamwork, communication and decision-making are vital in complex procedures.46 Numerous adverse events within the OR and emergency department are caused by human error and could be prevented by enhanced teamwork.46 Similarly, the endovascular suite is a complex multidisciplinary environment in which communication errors and equipment-related malfunctions have been shown to account for nearly half of all operative failures.47
Many of the CAS procedure rehearsals conducted at Imperial College London and Ghent University, Belgium are conducted as ‘whole’ team rehearsals and involve the interventionist, scrub and circulating nurse, and anaesthetist who are present in the subsequent real intervention. In this respect procedure rehearsal can be considered a powerful and comprehensive team training tool. These rehearsals are not solely focused on the technical elements of the procedure but also on training and evaluating non-technical skills. There is evidence that these non-technical and team interaction skills can be trained by complex, high-fidelity full team simulations48 and can have a positive effect on procedural outcome.48 Rehearsals to train the team can either be carried out in the laboratory environment, or in an authentic learning environment such as a simulated operating environment or real angiosuite (so-called in situ simulation) and so enhance contextualisation (see Figure 3). An example of such a high-fidelity simulation environment is the Simulated Operating Suite (SOS) at Imperial College London. The SOS is a replicated, fully functional, simulated operating theatre environment including all the necessary operative and recording equipment. Another example is ORCAMP (Orzone, Gothenburg, Sweden), a virtual angiosuite that allows integration of existing endovascular simulators and has been developed for training and assessment of the entire endovascular team.49 A drawback of these simulation environments are the financial costs and their limited availability. To circumvent this, simulators can be placed in the actual operating theatre or angiosuite, which was done frequently at the EVEREST centres. However, this kind of in situ simulation does rely on the availability of unused theatre capacity.
Patient Selection
Correct patient selection is paramount to ensure procedural and clinical success for CAS, since outcomes are influenced by the physician’s experience and patient-specific characteristics such as anatomy. Experts in the field now advocate a patient-tailored approach towards the CAS procedure.50 To meet this goal, in 2009 a scoring system was developed by CAS experts to evaluate the influence of anatomic factors on procedural difficulty44 (see Figure 4). The aim of this scoring system is to grade expected difficulty for CAS and guide inexperienced operators in proper case selection by identifying high-risk patients. The scoring system has been derived by expert opinion, using a Delphi consensus methodology. Anatomic characteristics of the arch and carotid arteries are graded from 1 (straightforward) to 9 (difficult). In the consensus document, 12 individual anatomic features were incorporated allowing for 96 combination anatomies. A scoring system for combination anatomy was produced, comprising broad agreement bands presented as traffic light colours – red for particularly difficult anatomy (score >7.0), amber for moderate difficulty (score 5.0–5.9) and green for lesser difficulty (score <4.9) The use of this scoring system may reduce peri-procedural stroke rates by the selection of patients appropriate to the operator’s level of expertise.
Recently this scoring system has been validated using patient-specific VR simulation.51 Novice interventionists performed three CAS cases of increasing difficulty, as defined by the scoring table (i.e. green, amber and red), requiring significantly more time to complete each case, with use of more fluoroscopy time, angiographies and contrast volume. More importantly, the quality of the procedure significantly deteriorated with increasing case complexity, falling below the arbitrary score of competent performance for the more difficult cases as measured by expert derived qualitative rating scales for CAS. These scoring systems, together with the use of VR simulation, may contribute to improved patient safety and outcome by enhancing pre-operative procedural preparation and identifying high-risk patients more effectively, especially when inexperienced interventionists are involved.
Discussion
Due to its minimally invasive nature, CAS remains an attractive procedure to reduce stoke risk in patients with atherosclerosis of the internal carotid artery. Nonetheless the initial enthusiasm has been tempered by evidence that peri-operatieve stroke risks may be higher with CAS than after CEA. This observation is partly attributable to inexperienced interventionists and their learning curve, inadequate training and suboptimal patient selection. Several strategies to improve performance and outcome after CAS exist. Apart from medical optimisation and CAS device refinement, patient selection, proficiency-based training and rigorous credentialing seem key factors associated with improved success after CAS. Incorporation of VR simulation into proficiency-based curricula for CAS seems paramount to increase physician experience with the procedure. Ongoing research in the field of simulation science indicates that this technology can enhance training and provide physicians with the necessary tools to increase their experience levels. Virtual reality ‘procedure rehearsal’ seems a promising adjunct in tailoring this training to specific patients.
National and international societies of the different subspecialties should now strive to create refined guidelines for CAS training, competency and credentialing. These will probably be more stringent than previously documented, as there is growing evidence of a steep and long learning curve associated with the procedure. In the near future, better trained CAS interventionists should be able to perform CAS to a higher standard and select patients more accurately, resulting in improved outcome and increased patient safety. CAS stroke rates may then become equivalent to CEA in specific patient groups, with the two strategies being complementary to each other in a patient-tailored approach to carotid stenosis and stroke treatment.