Research Report

INTERRATER RELIABILITY AFTER A BRIEF TRAINING ON CGI FOR PSYCHOLOGISTS

INTRODUCTION

Mental illness is one of the largest contributors to disability across the world.^1,2 The burden of psychiatric morbidity is increasing worldwide. The global burden of disease due to neuropsychiatric disorders was 6.8% in 1990 and is expected to rise to 15% by 2020. Among the ten leading causes of disability across the world, psychiatric disorders stand in the 5^th position.³ But for a population of 1000,000 the availability of psychiatrists is 0.4 and of clinical psychologists is 0.02. Only 0.25 mental health beds are available for 10,000 population in India.⁴

Improvement of the health system in any country requires research. In LAMI (low and middle income) countries, mental health research lack prioritization and there is paucity of studies in this area. The ‘9/10 gap divide’ in health research is acknowledged. Nine out of 10 publications in peer-reviewed journals come from high-income countries, where only 10% of the world’s populations live.⁵

Mental health research in India is lagging partially due to lack of expert manpower. Mental health professionals are overloaded with individual patient care. In India, there is no waiting list for consultants. All the patients seeking help are attended without delay. The same is true for the clinical psychologists also. When patient care demands immediate attention, overriding other functions, a mental health professional can hardly allocate any time for doing quality research. As a result, research in India in mental health is not getting its due importance.

There are several examples in India where basic / preliminary care of patients could be offered by trained lay persons where the professionals are not available. Health care manuals like ‘Where there is No Doctor: A Village Health Care Handbook’ by David Werner and ‘Where there is No Psychiatrist: A Mental Health Care Manual’ by Vikram Patel also are based on this assumption. The Accredited Social Health Activist (ASHA)workers of Kerala are a success story. Dr. Vidya Sagar practiced it in Amritsar decades back. The Sakalawara and Raipur Rani experiences, which paved the way for NMHP, are other examples.⁶ If clinical care can be provided by lay persons with training, why not the same in doing research also, especially in mental health?

In India there are 75 colleges offering BA degree in Psychology and 77 colleges offering MA degree in Psychology.⁷In Kerala, the figures are eight and two respectively and seven other colleges offer MSc in Psychology.⁸ M.Phil in Clinical Psychology [M.Phil.(Cl.Psy.)], Professional Diploma in Clinical Psychology [P.D. (Cl.Psy)], and Doctor of Psychology in Clinical Psychology [Psy.D. (Cl.Psy)] are the qualifications approved by the Rehabilitation Council of India (RCI).⁹ As per RCI, there are 21 colleges conducting approved courses in Clinical Psychology in India among which none are from Kerala.¹⁰ It can be assumed from these facts that there is dearth of clinical psychologists in Kerala, though general psychologists are available. Hence, training psychologists in the usage of tools for research is a feasible option.

The Clinical Global Impressions (CGI) scale is a standardized assessment tool. It is a valuable tool to rate the severity of illness, changes over time, and efficacy of medication, taking into account the patient’s clinical condition and the severity of side effects.¹¹

The CGI scale consists of three global subscales. The first subscale (severity of illness) assesses the clinician’s impression of the patient’s current illness state. The next subscale (global improvement) assesses the patient’s improvement or worsening from baseline. The third subscale (efficacy index) attempts to relate therapeutic effects and side effects by deriving a composite score that reflects both the therapeutic effect and the concomitant adverse effects.

In drug trials, the CGI scale is one of the most widely used scales to measure outcome. In the research field, CGI is an extremely useful tool. It can be applied over a wide range of psychiatric disorders. It has been found to correlate well with standard scales like Hamilton Rating Scale for Depression, Hamilton Rating Scale for Anxiety, Positive and Negative Syndrome Scale, Leibowitz Social Anxiety Scale, Brief Psychiatric Rating Scale, Scale for the Assessment of Negative Symptoms, and others across abroad range of disorders.¹² Therefore it can be used in lieu of the complex and cumbersome tools. The simplicity, brevity and the wide applicability are the advantages. It takes only 1-2 minutes to complete the scale. Time can be an important factor in a country like India, where there is deficit of psychiatrists. The average national deficit of psychiatrists in India is 77 %.¹³ So CGI becomes useful in the Indian scenario. There were reports that the scale is unreliable, contains redundant information and includes items that have abnormal distribution properties. It is opined that some of the CGI items are inappropriately constructed and are of doubtful clinical significance. The efficacy index has also been criticized. The anchor point descriptions on the subscale of therapeutic effects are also alleged to be unclear and unreliable.¹⁴ In spite of all these difference of opinions, the CGI scale is a well-established research rating tool applicable to all psychiatric disorders that can easily be used by the clinical practitioners.¹² The positive qualities of the scale could be enhanced by training and using of more highly structured anchor points for each item.¹⁵ Therefore CGI can be a useful research tool in the Indian context, despite its shortcomings.

It is in this context that we decided to conduct mental health research by imparting systematic and structured training to general psychologists who are not trained in clinics. We trained psychologists in the use of Clinical Global Impressions (CGI) scale in patients with severe mental illness (SMI) and measured the interrater reliability between them and with a trained psychiatrist. We defined severe mental illness as schizophrenia, mood disorders and any other psychotic disorder of more than two years’ duration.

METHODS

This study was conducted in a Government Medical College in Kerala as part of a larger study which assessed the gender differences of patients with severe mental illness. The study was approved by the Institutional Review Board (IRB) of the College. Written informed consent was obtained from the subjects.

Table 1: Inter-rater reliability of three investigators (I) on items of CGI

** Significant at 1% level; I1: Investigator 1; I2: Investigator 2; I3: Investigator 3

In this study, three psychologists who had completed two years course of M.Sc in Clinical Psychology were selected. Though their qualification was M.Sc Clinical Psychology, their clinical exposure was limited to four months internship training which was only an observership. Their degree has not been approved as a qualification of Clinical Psychology by the Rehabilitation Council of India. Therefore, we considered them as general psychologists for all intents and purposes. They were trained by two psychiatrists, for ten days, in interviewing skills, mental status examination and administration of tools used in the study. Their skills in interviewing and conducting mental status examination and administration of tools were supervised by the Principal Investigator (PI), the first author. PI is a qualified teacher of psychiatry with seven years post PG experience and has received training in administration of CGI. The patients who satisfied the inclusion criteria of severe mental illness were recruited by PI and written informed consent was obtained from them. We included patients, aged between 25 and 60 years, with schizophrenia, mood disorders or any other psychotic disorder of more than two years’ duration as severe mental illness. Six sets of ten patients with SMI were selected by convenience sampling. For the first set of ten patients, the interview and mental status examination (MSE) were conducted and CGI administered by investigators No.1 and No.2. For the second set of ten patients, the same procedure was repeated by investigators No.2 and No.3; and for the third set of ten patients, the same procedure was done by investigators No.1 and No.3. After this, for the next ten patients, the procedure was repeated with Principal Investigator (PI) and investigator No.1 and for the subsequent sets of ten patients each, with PI and investigators No.2 and No.3.

The interrater reliability between all three investigators (I¹ and I², I² and I³, I³ and I¹) and each investigator with the PI were measured using Spearman’s rank correlation coefficient.

RESULTS

The interrater reliability between the investigators was high in the use of subscales 1, 2 and 3 of CGI. The correlation was least between I1 andI3 in subscale 1(Table 1).

With regards to interrater reliability between the investigators and PI, variable responses were received. In CGI 1, there was significant correlation between the PI and all three investigators. In CGI 2, only the correlation between PI and investigator No. 3 was significant. In CGI 3, the interrater reliability exists between PI and only investigator 1 (Table 2).

Table 2: Inter-rater reliability of Principal Investigator (PI) and 3 Investigators (I) of Items of CGI

* Significant at 5% level; ** Significant at 1% level; I1: Investigator 1; I2: Investigator 2; I3: Investigator 3; PI: Principal Investigator

DISCUSSION

This study highlights the usefulness of training psychologists who have no clinical background in the use of scales which can be used for research. We found significant interrater reliability between the investigators and between them and PI as regards item No.1, i.e. the severity of illness. Regarding the 2^nd item of CGI, which measures the improvement in symptoms entirely due to drug treatment, a short training is found to be inadequate. Regarding the 3^rd item, the side effects of drugs, there was no correlation between the ratings of PI and the investigators. This is understandable considering the fact that a psychiatrist, a person with medical training, would be better able to delineate the side effects from the multitude of complaints the patient presents. A psychologist will be unable to appreciate and rate it as they are without a medical background. Both 2^nd and3^rd items of CGI involve association with drug effects, but the first item about the severity of illness, “Considering your total clinical experience with this particular population, how mentally ill is the patient at this time?”, takes into account only the gravity of the illness. The reliability was significant in the first item which takes into consideration the clinical experience of the clinician. Even after a brief training, the investigators had significant interrater reliability with a psychiatrist of nine years of experience trained in the use of CGI. This indicates that with substantial training, general psychologists may be actively included in mental health research, with guidance and supervision, in a country with deficient psychiatrists and poor mental health research output.

LIMITATIONS

We had imparted training only to three psychologists as part of the study. The period of training was only ten days.

CONCLUSION

The fact that there is significant interrater reliability in the item “severity of the illness”, which assesses the clinician’s impression of the patient’s current illness state, but not with the other items of CGI shows that the services of trained para professionals can be utilized in research, with certain restrictions. More intensive training would be more effective, and certain subscales which involve the side effect of medications would be better administered under supervision of psychiatrist. Nevertheless, this study throws light on how a resource poor country can make use of paraprofessionals for doing research work by imparting adequate training, though with certain limitations.

REFERENCES