spacer
home > ict > autumn 2011 > online lessons
PUBLICATIONS
International Clinical Trials

Online Lessons

Sabine Veigel and Andreas Grund at GCP-Service International provide a comparison of training methods for site staff, including an analysis of face-to-face versus online training methodology

Conducting a clinical study is only justified when it has been appropriately planned, implemented, evaluated and reported from ethical and scientific standpoints. In the field of clinical research, national laws, international guidelines and their implementation have been subject to constant development over the past few years.

Germany has particularly strict requirements in place in terms of investigator training courses (1). German ethics committees require that all investigators attend a one or two day training, which covers national regulations, as well as ICH GCP (2). These intensive training activity demands are criticised quite frequently by investigators and sponsors, although this requirement has not been invented by German ethics committees themselves, but is required according to ICH GCP (ICH GCP 4.3.1) (3).

It is expected that investigators as well as other site staff, sponsors and CRO personnel involved in clinical studies have the expertise to ensure the correct conduct of clinical studies. For this reason, both the pharmaceutical and medical device industries spend a lot of time, money and resources for training personnel, investigational sites and CRO staff. Whether these investments are an effective way of building up required competencies is rarely questioned. Often, a lot of time and money is spent on long seminars and face-to-face training. However, from educational science we know that these training methods are only successful for a limited number of people. When participants are asked about the content of the training course they attended a few weeks later, the results are often disappointing (see Figure 1).

Most course participants need learning methods with a more systematic approach and regular revision as well as testing of the newly acquired knowledge. This enables participants to retain the learnt material over a longer period of time and successfully apply it in practice.

METHODOLOGY

Recently, a prospective survey was designed in order to compare a lecture-based GCP course with an e-learning GCP course of identical content. The participants were investigators who had completed face-to-face training on GCP topics and a participant group, who had completed an online GCP course with identical contents. Both eight-hour training courses have been accredited by the Medical Association of Bremen (Germany) and include the following topics:

  • Introduction: development of drugs and medical devices
  • Basics of biometrics: different study designs and sample size calculation
  • The declaration of Helsinki and the GCP guidelines
  • GCP definitions/important GCP documents
  • Investigator responsibilities part one
  • Investigator responsibilities part two/sponsor responsibilities/audits
  • Inspections and the reporting of safety-related events
  • Current legislation and regulation on clinical trials in Europe

In order to receive a training certificate from a lecture-based course, the Medical Associations require that the success of the training is measured. In this case study, the test contained 15 questions about the training content, which the attendees needed to complete immediately after or during the training. The 15 questions cover all eight topics of the training. Investigators have been trained face-to-face in the eight training modules between 2009 and 2011.

In 2009 a training course was developed alongside an online equivalent. The attendees of the online course received an individual username and password which must be kept confidential. Each of the online training modules consisted of an animated video presentation that took approximately 45 minutes. Afterwards a multiple choice test needed to be passed for each module. Each test consisted of 15 questions that were generated randomly from a pool of 40 questions, and 80 per cent of the questions needed to be answered correctly to pass the test. If a trainee failed the test, three test repeats were possible. With each attempt, new questions were generated. The attendees had access for two months, in which all eight tests needed to be passed to receive the training certificate automatically. Technical as well as GCP-related questions could be asked by email, and the trainees received an answer within 48 hours.

The statistics of the online training platform and the results from the face-to-face test could be used to compare shortterm test results in both groups to evaluate the advantages and disadvantages of each training intervention. The t-test was used as a point of comparison if differences were significant. The number of attendees varied between 17 and 128 for the online training and 58 to 578 in the face-toface training, because not all attendees gave feedback. To compare the groups the percentages per group were used.

Furthermore, investigators were given the opportunity to complete evaluation questionnaires consisting of quantitative components and open-ended qualitative responses to assess different parameters of the training courses in respect of acceptance and satisfaction of the attendees with the respective course. The following parameters were evaluated: degree of satisfaction; usability; intellectual approach; failure rate; duration; practicability and direct/indirect costs. By calculating the potential travel expenses based on the places of residences of the online training participants, the courses could also be compared in respect of their costs.

RESULTS

The first result from the survey was that the increasing requirements for training in GCP and the national requirements correlate with the number of training attendees. Figure 2 shows the development of training attendees on the two different courses from January 2009 until August 2011. In both courses the total number of attendees increased significantly, with online attendees growing the most.

In terms of the satisfaction of the attendees, the results demonstrated overall satisfaction with both training methods. The usability of the course contents in professional practice was regarded as ‘high’ or ‘very high’ by 68 per cent of the investigators. Eighty-six per cent of online training participants and 94 per cent of face-to-face training participants who completed the evaluation form rated the GCP training as ‘good’ or ‘very good’, which results in an overall grade of 1.8 for the online and 1.5 for the lecture-based course (see Figure 3).

On a scale from ‘too difficult’ to ‘too easy’, the course contents regarding the lectures were evaluated as ‘just right’ by 71 per cent of the online course participants and 91 per cent of the lecture-based course (see Figure 4). The practical examples used were found as right by even 100 per cent of investigators trained online and by 91 per cent trained face-to-face (see Figure 5).

The practicality of both courses has been judged very similar. Both groups of attendees appreciated the practical examples used in the courses to explain certain rules and requirements.

As results from the open questions answered by investigators trained online show, the flexibility of the online training programme was very much appreciated. Expectations of the online training participants could be met particularly in terms of the quality of content (grade of 1.7), professional benefit (grade of 1.8) and examples of use (grade of 1.9). However, online participants reported that there was too much content for the allotted timeframe of 45 minutes per module.

In order to determine the total period of time needed by the participants to complete the online training, the dates of the test visits of the investigators in the statistical database of the online platform ILIAS were analysed. By means of the time difference between test visit one (first visit) and test visit eight (last visit) an average of 11.45 days (0 to 39 days) could be calculated. This analysis does not acquire the net working hours, but the absolute timeframe in which the training was performed. Three participants completed all eight modules of the online training within one day, another three even within hours (0 days), whereby the shortest processing time lasted nine hours, 28 minutes. The average duration of each online training test covering 15 questions was 14 minutes and varied between 10.8-19.9 (±8) minutes. All investigators trained online or face-to-face passed the required tests. On average, 87 per cent of the tests questions were answered correctly in both groups.

In addition to the time the costs were compared, the direct as well as the indirect costs (see Figure 6). The price for a face-to-face training session is €400, which reflects the mean price for one day face-to-face GCP trainings in Germany. The price for an online GCP training is €350, which makes a difference of 12.5 per cent (p<0.05). The difference by taking into account the indirect costs was even higher. There are no indirect travelling costs to online training, whereas the estimated travel costs for the travel to the training are €263.4 (p<0.01). If attendees needed to stay overnight the costs increased, but these additional costs have not been considered. The overall cost difference between both methods is €313. Furthermore, the loss of pay for selfemployed investigators is significantly higher for the face-toface training, which took place during the week compared to the online training, which was usually done in the evening or during weekend.

Considering that 58 per cent of the investigators had ‘little’ or ‘no experience’ with new online media, the majority responded that they felt comfortable in using this course format and would prefer the online training over classroom training. According to the participants, the main strengths of online training include the free timing, no travelling effort, lower costs, convenient learning conditions in personal environments, the possibility of repeating individual chapters and selfdetermination of the amount of subject matter or speed of learning. Potential disadvantages mentioned include the lack of communication with experts and with any group of other participants, a need for free time, technical prerequisites and self-motivation/ discipline needing to be raised.

DISCUSSION

The ever increasing scientific complexity of clinical studies has enlarged the number of study patients as well as the number of study sites during the last couple of years. In parallel, the regulatory complexity has changed the responsibility of sponsors, CROs, ethics committees and competent authorities. A few years ago research personnel needed to have simple ICH GCP certificates in place, which was sufficient to comply with the expectations of the inspectors and ethics committees. The method of gaining the certificate was usually not scrutinized; short GCP courses were conducted during investigator meetings. However, with the implementation of the EU Directive 2001/20/EC, and thereby ICH GCP, into the national legislations of the EU member states, the requirements became stricter (4).

Nowadays, clinical research personnel do not need to prove training attendance; instead the focus is on competence in research-related topics. This is refl ected by the fact that most of the inspection and audit findings reported by the authorities and sponsors on a regular basis refer to deviations against ICH GCP, as well as national regulations. Hence, training in ICH GCP alone cannot be considered sufficient to ensure data integrity, or the rights and welfare of study subjects. The consequence of this is that the training efforts spent for clinical studies exploded. The common way of providing training to research staff cannot be considered adequate as it is not possible to train such a high number of new staff in a systematic and cost-effective way. Furthermore, the fl uctuations in site staff is quite high in clinical research, which increases even more the efforts, costs and energy that sponsors need to conduct training.

The evaluation of the two different training methods showed that both methods have advantages and disadvantages. Both training methods were much appreciated by the trainees, who liked the content of the training modules. The satisfaction grade of face-to-face training is not significantly higher than online training. The conditions for obtaining a training certificate for face-to-face training are easier to meet as only a limited number of test questions are required, which can even be answered during the training. For online training, a much higher number of randomised test questions needs to be answered correctly, which also extends throughout the duration of the training until a certificate is obtained. Nevertheless, the total time from registration for the online training until obtaining the training certificate is 11.45 days on average, which is extremely fast, and usually lasts much longer for face-to-face training, as the trainee is dependent on the date and location of the training course. Cost reduction is another main advantage of online training. According to the results of this evaluation, online training is about €300 less expensive per trainee compared to face-toface training. For midsize studies, on which 100 investigators (principal investigators and sub-investigators) are involved, the cost difference would be €30,000. Hence, online training is an economic way of providing knowledge, saving time as well as direct and indirect costs through reduced travel expenses. This advantage has generally been recognised in medicine, which caused an increase of certified online training in medical topics of 41.9 per cent between 1998 and 2009 as shown in Figure 7 (5). The new technologies now available make use of well-developed GCP online training courses covering scientific backgrounds, as well as national regulations, resulting in a logical consequence for the clinical research business.

CONCLUSION

Taken together, this observation study has shown that both methods seem to have the same learning effect in the short term. These results have already been shown in other studies; from an economic point of view, online training has significant advantages as the costs per investigator are significantly reduced (6). The continuously and steeply increasing number of online attendees also indicates that a web-based training form is very much welcome by the trainees and the advantages are well recognised.

Whether the long-term learning effect of both training methods is comparable needs to be evaluated in future. However, it can be assumed that the learning effect of an online training, which takes place over several days when combined with tests, is at least not less beneficial for the memory of investigators compared to a one day crash course, as it is known from different educational areas as well (for example, learning of languages).

References

  1. German Medical Association, Announcement of the German Medical Association, Recommendations by Ethics Committees in clinical trials for evaluating the qualification of investigators and the suitability of trial sites according to German Drug Law, Dtsch Arztebl 107(1-2): A48-A51, 2010
  2. Bergtholdt B and Gaertner U, Investigator and trial site qualification – the requirements are becoming more severe, DGPharMedNews 12(3): pp115-117, 2010
  3. ICH Topic E6(R1), Guideline for Good Clinical Practice
  4. Directive 2001/20/EC of the European Parliament and of the Council of 4 April 2001
  5. Mack J, Pharma Support for Accredited CME Continues to Decline, Pharma Marketing News, 2011
  6. Cook DA, Levinson AJ and Garside S et al, Internet-based learning in the health professions: a meta-analysis, JAMA 300: pp1,181-96, 2008

Read full article from PDF >>

Rate this article You must be a member of the site to make a vote.  
Average rating:
0
     

There are no comments in regards to this article.

spacer

Sabine Veigel (MSc) works in the clinical research area since 2010, when she carried out her Master’s thesis on the topic ‘Scientific evaluation of a GCP online training’ at GCP-Service International as part of her studies in Nutritional Science at the University of Giessen. Since May 2011 her main responsibility within GCP-Service International is working as Clinical Research Associate and Online Training Manager. Email: sveigel@gcp-service.com

Andreas Grund (MSc, PhD) has worked since 1997 in the clinical research sector. He worked in clinical monitoring, quality assurance, project management and training for the University Kiel, and global CROs. In 2004 the CRO GCP-Service International was founded by Andreas in Bremen (Germany). Currently his main responsibility within the company is consulting and training of pharmaceutical and medical device companies in regulatory requirements, national laws and international guidelines. Email: agrund@gcp-service.com

spacer
Sabine Veigel
spacer
spacer
spacer
Andreas Grund
spacer
spacer
Print this page
Send to a friend
Privacy statement
News and Press Releases

PHASTAR Announces Investment by Charterhouse Capital Partners

PHASTAR, a specialist biometrics clinical research organization (CRO) for clinical trials, announces an investment made by Charterhouse Capital Partners LLP, a long-established European private equity firm, which will support PHASTAR’s growth in biometrics and geographic expansion. Charterhouse is making its investment in partnership with PHASTAR’s Founder and CEO Kevin Kane, who remains a significant shareholder in the business. Kevin and the existing management team will continue to run, manage and operate the business on a day-to-day basis.
More info >>

White Papers

The Key To Tableting Success: How To Choose The Right Tooling

Natoli Engineering Company, Inc.

Choosing the right tooling can increase tablet output, decrease waste and ultimately determine the success of a product launch.
More info >>

 

 

 

©2000-2011 Samedan Ltd.
Add to favourites

Print this page

Send to a friend
Privacy statement