|
CT FEAT: Families Helping
Children Achieve Their Full Potential |
 Print
This Article
|
|
How Can We Judge Autism Treatment Claims?
By Richard Irwin (CTFEAT member)
Thus far there is no one universally accepted and recommended treatment
for autism. That is not to say that all treatments are equally
effective. Deciding which treatment is appropriate for your child can be
an exceedingly difficult and stressful choice. The first question that
must be answered before making this choice is: What standard should we
use when evaluating treatments for autism? A treatment can only be
deemed effective if it is based on sound, scientifically validated
principles and supported by empirical data. In simple terms this means
that treatments for autism must be backed by the same quality of
research that we demand from other fields of science, such as medicine,
chemistry, and engineering.
What questions should we ask when evaluating treatment claims?
Unfortunately, parents are most often left on their own to read the
research literature and draw their own conclusions as to the
effectiveness of a particular treatment. The best allies a parent has in
their search for a treatment for their child are a healthy dose of
skepticism and a willingness to ask questions. The following is a short
list of questions that should be answered when evaluating claims about a
particular treatment. For a comprehensive guide to evaluating treatment
claims, the reader is referred to the chapter "Evaluating Claims about
Treatments for Autism" in Behavioral Intervention for Young Children
with Autism, edited by C. Maurice, G. Green and S. Luce, (Pro-Ed, 1996).
What were the goals of the treatment program?
Children with autism exhibit cognitive, social, behavioral and
communicative deficits. A treatment program designed to focus on only
one or two narrowly defined areas shouldn’t be considered sufficient.
For instance, the goal of a proposed treatment program may be only "to
increase the use of language." However, even if language usage improves,
that would not necessarily remediate the other core deficits typically
associated with autism. A child can make tremendous gains in his speech
and still exhibit severe behavioral, social and cognitive problems. A
treatment program should address all the core deficits found in children
with autism.
Some programs appear to subscribe to the philosophy that "the best we
can do is manage the child’s maladaptive behaviors." Such programs often
set up very structured environments and routines to minimize troublesome
behaviors, but accomplish little else. These programs underestimate the
enormous gains in every domain that many children can make with
effective intervention.
Were there specific measurements made pre- and post-treatment?
Very often, words such as "good to outstanding" or "significant" are
used when describing the gains made by autistic children in a treatment
program. While these descriptors sound good, in reality they tell us
nothing about the effectiveness of the treatment unless they are defined
in advance and based on data derived from specific measurements. There
are currently many accepted standardized tests that can be used to
measure the progress of autistic children in a treatment program. Some
examples are the Intelligence Quotient (IQ) test, the Vineland Adaptive
Behavior Scale, the Childhood Autism Rating Scale (CARS), the Peabody
Picture Vocabulary Scale and the Personality Inventory for Children.
These tests are used to measure such things as cognitive function,
language development, adaptive behavior and social skills. As children
with autism spectrum disorders exhibit a broad range of deficits, a
combination of these tests should be used to establish base lines and
document progress. A meaningful determination of progress can only be
made if these tests are administered before and after treatment, and the
results carefully compared.
One common but misleading measure used to demonstrate the effectiveness
of a treatment program is the percentage of its children that transition
to "regular school placements." But this measure often has very little,
if any, significance. A child meeting this measure of "success" could
require extensive supports to function in a regular school placement -
or none at all. The child's "regular school placement" could be due more
to local school policies - such as mandated "full inclusion" - than to
the child 's progress.
Who made these measurements and were they potentially biased?
Tests such as those mentioned above should only be administered by well
trained, experienced professionals. Steps should be taken to minimize
the possibility that the data collected is biased by the desires of the
investigator to prove treatment effectiveness. One method of eliminating
investigator bias is to have those administering the tests and
collecting the data be completely independent of those administering the
treatment. Those performing the tests should have no knowledge of the
treatment program or its predicted outcomes. This is referred to in the
literature as a "double blind" model. Some measures, such as parent
testimonials or satisfaction ratings, can be biased and should only be
one part of a comprehensive evaluation of a program. It is also wise to
be skeptical of professionals designing, researching and evaluating
treatment studies outside their area of expertise. For instance, we
wouldn’t want a podiatrist developing and reporting on a new dental
procedure.
Was the progress made due to the treatment?
While this seems like a silly question, it is necessary to demonstrate
that the progress made is due to the treatment and would not have
occurred in the absence of the treatment. One method used is to collect
data on two identical (all test measures equivalent prior to treatment)
groups of children, one group receiving the treatment in question and
the second group not receiving the treatment. Differences in progress
between the two groups can then be attributed to the treatment under
study. There also must be controls to ensure that no other plausible
explanation for the results exists. No matter the approach used, there
should be some explanation of the experimental controls used and why
they were appropriate. One type of study of a treatment’s effectiveness
that is fairly common in the autism literature is the "chart" review. A
chart review is a summary of the data and clinical observations taken
from patient or student records. In general, a chart review does not
contain experimental controls and should be viewed with skepticism.
Usually chart reviews are used to identify potential trends that can be
investigated experimentally.
Were the gains permanent?
A treatment can only be considered effective if the gains made are long
lasting and permanent. The best method to determine this is to perform
follow-up testing, employing the same measures used in the treatment
study, at intervals substantially after the discontinuation of the
treatment.
Were the results of the study reported in a reputable scientific
journal?
The vast majority of reputable scientific journals are peer-reviewed.
This means that before an article is published, the article is reviewed
and commented on by other experts in the field. While this by no means
guarantees the validity of the data reported, it does provide some
measure of scientific scrutiny on the quality of the science published,
the interpretation of the data and the claims made. Articles published
in newsletters, flyers or books are generally not peer-reviewed, and so
the potential for inflated or false claims is greater. Practically every
day we see flyers that make ridiculous claims about weight loss
procedures or hair growth. We do not accept these at face value and the
same skepticism should be applied to treatment claims for autism.
Have the results been reproduced?
All scientific data should be viewed with skepticism until it has been
reproduced by independent investigators. As an example of why this
should be the case, we only have to consider recent history. Some years
ago, two scientists published a report claiming that they had achieved
cold fusion - essentially the ability to produce nuclear energy from
simple water. This energy source would be clean, safe and inexhaustible.
The news was so exciting that the nation and worldwide media reported on
it immediately. However, when other scientists attempted to reproduce
the original findings, they found they couldn’t. There had been errors
made in the original study and it turned out that they had not actually
achieved cold fusion. We do need to be aware that studies in the field
of autism are generally more difficult to reproduce than are those in
some other fields of study. Besides requiring an ample number of
autistic children and the potential ethical questions involved, these
studies require enormous resources, time and trained personnel to
complete.
Where can I turn for help?
The families of CT FEAT have each gone through the process of choosing a
treatment program for their children. If you would like to learn from
their experiences, please feel free to contact CT FEAT by phone at
860-571-3888 or by E-mail at
ctfeat@ctfeat.org.
Back to Top
|
|
CT FEAT, P.O. Box 7063 Prospect, CT 06712-0063
Copyright 2022
CT FEAT
CT FEAT
Contact Us
|
|
