|
|
Reading
Centre
A Reading Centre is staffed in the following areas, a lead clinician,
principal investigators, senior graders, graders, graders in training,
epidemiologists, statisticians, photographers and other staff as needed.
The lead clinician and principal investigators are ophthalmologists.
Grader
Training
Graders have no medical background, but are usually educated to degree
level. A senior grader trains them until they perform to a certain standard.
Graders must sit a grading exam at the end of their training. Overall
training is for a period of between four and six months with additional
specialised training sessions. For a grader to receive certification they
need to grade a certain set of exam photographs, where grading has been
agreed by the senior grader and the reading centre advisor. They need
to grade this set at least twice with a 14-day gap in between. The grading
is then entered onto a database and intra and inter-observer agreement
are calculated. If this is acceptable a grader can grade alone with some
supervision. If not acceptable, further training is followed by re-examination.
Graders are generally certified to grade for a specific disease and not
as a grader. Currently the problem with grader training is that no universally
accepted training schedule exists. In addition there is the question of;
who certifies graders in areas when there is no accepted grading form/severity
scale.
Logistics
After the protocol has been agreed photographs are taken and sent to The
Reading Centre. On their arrival the images are checked and catalogued.
They are then kept under lock and key for the entire time that they are
held at the Reading Centre. Graders do not have access to the photographs
and are given the images on a daily basis by the data handler. Once graded,
images are given directly back to the data handler and then once again
locked away along with the completed graded forms. As
images are always double-graded this process ensures that no grader has
access to previous grader results. The data collected is then entered
into a secure database which is stored on a separate drive, no-one other
than the data co-ordinator and IT staff have access to this drive.
Quality
Control
For quality control purposes a subset of photographs from each study is
double/triple graded and entered in to the database. Again, 14 days has
to elapse before a grader is allowed to regrade images they have previously
graded. This ensures that graders will be unable to remember or memorise
any of their prior grading results. Subsets of photographs are graded
by the same person, entered, analysed and discussed. A separate set of
photographs are kept for temporal drift, they are graded every year, entered
and never discussed in detail. If results of this set are not up to standard
then further tuition is given.
A major problem with quality control is that all reading centres currently
do their own. This makes it very difficult to assess studies. There is
a push for specific quality control procedures or image 'swapping'.
Intra and inter observer agreement can be described in three ways; exact
agreement, kappa values and weighted kappas. When presenting these to
the graders, specific explanation is given about how they influence the
overall results.
Protocol
and Time
When a protocol is developed for grading it needs to be categorically
decided and stated what we are to be looking for when we grade. Points
to consider are; need to be able to answer the proposed question, needs
to be specific enough and needs to be quantifiable and reproducible. Most
importantly, a study's needs have to be matched with the centre's capabilities
and needs to be completed within the given time-frame. A rough guide to
how many images can be graded within a specific time are given below:
Experienced Grader; 10 AMD photos/one hour, 3 Diabetic Retinopathy/one
hour, 3 FFA/one hour.
Not Experienced Grader; 3 AMD photos/one hour, 1-2 Diabetic Retinopathy/one
hour, 1 FFA/one hour.
For concentration purposes and for maximum accuracy a grader should only
be allowed to grade for between 4-6 hours a day.
Problems
with Grading for Different Studies
In general there are still problems that continue to occur with grading
for different studies. As yet, there is no universally accepted grading
form and no universally accepted fundus photography system in existence.
It is also a concern that there is no official programme for grader training
and quality control. When grading for screening purposes, such as diabetic
retinopathy, there is no information if high-risk or end stage disease
patients are seen or not. These all need to be kept in mind when analysing
grading results.
Summary
To summarise the process of grading an important point is to aim before
you grade. Make sure that you have taken all aspects of the study into
consideration and clarified anything that is not clear; it is imperative
that a clear and concise protocol has been developed and agreed upon.
Make sure that you have your logistics right. Detailed and structured
grading is essential. Finally, be certain that you have your quality assurance
correct.
Data
Management
Why have a database?
All organisations rely on the importance of good data and its availability.
A database is basically a filing cabinet for that information which can
be accessed via networked computers, intranets and the intranet. It means
that information is readily available at the touch of a key. It also means
that you are able to be more selective about the information that you
retrieve.
Managing a Clean Database: Rubbish in, Rubbish Out
A lot of us are familiar with the saying or expression; 'if you put rubbish
in you will get rubbish out.' If rubbish is entered into the database,
this will obviously have a great effect on what comes out. Rubbish, obviously
being inaccurately entered information, will undoubtedly have a huge effect
on the final results of any study taking place. If data is entered badly,
the results of the study will be inconclusive. If this happens and data
is not double or even triple entered or checked, the likelihood is that
the whole procedure will have been a complete waste of time, effort and
money. Making data entry as simple and as straight forward as possible
is the easiest way to prevent bad data being entered.
Designing a Clean Database - How Easy is it to Enter
Data?
Drop Down Boxes
A drop down box works, by giving the data entry person the choice
of codes, numbers or words relevant to a particular field. They are useful
to doubly confirm a field is entered in the correct manner. Although it
is difficult to enter data in this way, as it is very time consuming.
It does restrict the opportunity for error, by limiting the available
keying options to those stored in the list.
Locking
the Database
Locking the database ensures that each field is completed in the correct
manner. Although this does not - in any way - guarantee that fields have
the correct information entered.
Matching Tab Order
Tab order in the database should match the running order of the data sheet,
from which data is being entered. This assists the person(s) entering
the data to obtain some sort of running order in which data is keyed.
Data entry then becomes consistent, which minimises the risk of mistakes.
Do not 'crowd' the page
If the database has been designed well then a page should not be crowded.
By keeping a database sheet clean and simple, it makes it easier for the
data entry person to scan for errors after each sheet has been completed.
One database screen, per datasheet.
Data is much easier to enter if the database screen is kept to one sheet
only. Anything more than this greatly improves the opportunity for errors.
Automatic Wording or Coding
An example of this is, if a word or code needs to be repeatedly entered,
programme likely words into the database so that it completes the word
after the first letter is keyed. For instance, if on every database sheet
a field needed to be completed with a person's name, the database could
then automatically make the appropriate selection after the first one
or two letters is keyed.
Summary
We can see now the overall importance of a well-designed database and
how careful planning can greatly assist in achieving and providing accurate
results. When considering your database design you should also have clearly
defined study objectives and outcome measures. You should consider which
statistical method you will be using for analysing your data and ideally
meet with a statistician beforehand for some practical advice. Be aware
of the data availability from your data-collecting source and confirm
that all necessary fields will be able to be completed. Entering a small
amount of data as a test in the beginning of a study can save a lot of
time and heartache in the future. Therefore it is essential to regularly
monitor the data and feedback, giving you the opportunity to deal swiftly
with any problems or to make any necessary amendments to the database.
|