Evidence-Based Education

I. The problem

Concern exists about the levels of academic achievement attained by students within the American public education system.This is especially true for students in special education programs. In an effort to increase academic performance and accountability, the United States Department of Education is transforming the basis of educational practice. Evidence-based, or scientifically-research supported, educational programs are the cornerstone of the Bush Leave No Child Behind Policy. The challenge, stated Undersecretary of Education and Rehabilitative Services, Robert Pasternack, PH.D., (June 6, 2002, Committee on Educational Reform) is to use science and evidence to guide policy and instruction.

II. What is evidence-based education

Evidence based education is based in principles of scientific inquiry. These methods have been common to the fields of biology, psychology, economics, and other areas of science. These principles can also be applied to education. For example, does one method of instruction improve academic achievement more than another? Do some instructional methods work better at different stages of child development? Is one method better for a specific subject area? Answers to these questions suggest a better tuned educational system that can assure higher academic achievement for our youth.

III. Principles of scientific inquiry

• Pose significant questions that can be investigated empirically.
  • Does Reading Program A increase reading achievement significantly more than Reading Program B? This question can be scientifically investigated.
  • Society causes reading problems. This is a very broad topic with many variables. It would be difficult to investigate.
• Link research to theory
  • Take proven models of child development and link an educational program to it.
  • Children taught using an active model of instruction (hands-on based on Piaget’s theory of child development) will achieve at a higher level that children taught using a passive model of instruction (grade reinforced- based on Skinner’s model of learning)
• Use methods that permit direct investigation of the question.
  • Methods vary in their ability to provide understanding of what caused the increase in academic gains.
  • Observation of children can give ideas but a comparison of standardized test scores can be statistically analyzed to see if one group of children does better with one method of instruction over another—assuming all other factors are held constant.
• Provide coherent chain of rigorous reasoning.
  • Clearly state the assumptions
  • Present the analysis from the data gathered
  • Tell how the evidence is judged to be relevant
  • Tell how the evidence relates to theories
  • Define levels of error that may be present in the current research.
  • Offer alternative explanations.
• Replicate and generalize
  • If the same research study is done again with another group of children can I expect the same results.
  • Can I generalize the findings to most children in school. For example, Do the results of this research show that program A will work better than Program B in First Grading reading classes?
• Scholarly debate
  • Are the methods clear and open to review by others who work in the field
  • Are the results published in peer reviewed journals. For example, is the research found in the Journal of Learning Disabilities where a panel of experts in the field of LD has reviewed the study or is it reported in a popular parent magazine with no evidence of expert peer review?

IV. Methods

Rigorous scientific inquiry requires methods of research that allow the prediction of cause and effect. If Reading Program A is used rather than Program B or C will more significant gains in academic achievement among first grade readers be achieved? This assumes that all other factors will be equal, or as equal as the researcher can make them. For example, the children in the class will all comes from schools with similar socioeconomic status and teacher/pupil ratios, among other things.

It is important to know how to evaluate research. Suppose a researcher studies reading achievement in a predominantly inner city school where factors such a limited parent income, limited resources, etc. result in generally poor reading achievement scores among a schools population of students. If the researcher demonstrates increased reading achievement with a specific reading program, an evidenced based approach will also want to look at what factors produced the gain. It could be increased teacher time with the child, more parent concern, increased funding sources for the school, or the materials presented in the specific reading program.

If the researcher compares reading achievement results using two different reading programs at the same school and finds that one increases achievement significantly more than the other, it is more realistic to conclude that the reading program is related to increased reading achievement.

Does this mean this program will work well with all children? Not necessarily. To demonstrated that it works well with students who have high and low cognitive ability the researcher would compare these groups. To demonstrate that the reading program was equally effective for students with LD and students in the general education program, the researchers would compare the standard reading achievement scores of students with LD and those without LD. Limited research exists to document the effectiveness of specific programs for students with learning disabilities and/or ADHD.

When looking at research studies it is very important to look at the type of research presented, the variables involved and the number of subjects or individuals involved in the study. Here are some important factors to look at:

1. Qualitative or quantitative?

   Is the research qualitative (based on observations, questionnaires, discussions) or is it quantitative (based on empirical data such as standardized test results)?

   1. Qualitative

      The researcher watches Billy’s behaviors and determines what upsets him when reading. He may observe that the teacher did not call on him because he seems anxious and nervous as she looks at him. Does this mean all students who are not called on in a reading class will be upset? No. This can give an idea of something to look at in reading achievement but it does not allow one to predict cause and effect.

      Good practice would not be to suggest modifications in a school’s reading program based on the analysis of Billy’s behavior and his low reading achievement scores. Good practice, though, might include modification of Billy’s classroom procedures based on the qualitative behavioral analysis of factors that cause him frustration. The teacher might want to alert Billy that she will be calling on him the next day by giving him a question to think about so that he feels more secure in front of his classmates when she turns to call on him.

      Qualitative research can generate questions or hypotheses from which controlled research designs can evolve. These are often referred to as quantitative research since data analysis includes scores earned on tests- such as tests of achievement.

   2. Quantitative

      Quantitative analysis will not give us much information about why Billy is upset with reading but if gives clues about a general population of children in school. It can tell us if one program works better to increase reading achievement among a population of students.

      

      For example, comparison of the standardized test scores of 1000 children in first grade comparing three reading programs is a quantitative analysis. All groups of children are from comparable school systems. Achievement is measured using standardized scores from a valid and reliable reading test that had a mean of 100 and a standard deviation of 15. Reading skills tested were word recognition (WR), sentence comprehension (SC), and passage comprehension (PC).

      In this design there is quantitative data that shows the differences in achievement outcome based on the three different models of reading instruction. These are shown in Figure A above. This research would lead to the conclusion that Program A leads to increased reading achievement in all areas measured when compared to program B and C.

      Based on the conclusions from this research it could be anticipated that funds spent to introduce Program A to children in public schools will increase reading achievement.

2. Is the study correlational or does it use a randomized controlled research design?
   1. Correlational study

      Correlational research looks at how two factors goes together. For example, the longer students are in Ms. Jones class the better they learn to read. Ms. Jones uses Reading Method B from the study above. It is tempting to conclude that increased exposure to Reading Method B results in increased academic achievement but it could also be from the teacher’s personality or some other factor in the classroom. This does not prove that Method B is better.

   2. Randomized controlled experiment.

      In this type of study students are randomly assigned to different reading groups that use either Method A, B, or C. All other factors are controlled. Researchers will have many different teachers using these methods. The children will come from similar backgrounds, and the length of time the Methods are used will be the same for all children. The only important variable will be the type of reading program.

      In this case, data indicates that one program is superior. If the groups of children in the study a significant difference in academic achievement between children using Method A, B, or C will be reported.

      The randomized controlled research design will be central to the new mandate for research-based education policy and instruction.

V. Conclusion

Evidence-based education will become the standard. School systems will be expected to use educational programs in core subject areas such as math, reading, social studies, etc., that have proven success in enhancing the measured academic achievement of school-aged children.

To learn more about scientifically based research from the US Department of Education’s conference on this topic.

Posted June 4, 2002.

VI. About the author

Kathleen Ross-Kidder, Ph.D. is the Director of LD OnLine (www.LDOnLine.org), the interactive guide to learning disabilities for parents, teachers and other professionals, and the leading Web site in its field. LD OnLine is a service of WETA-TV-FM, the public broadcasting station in Washington, D.C. Dr. Ross-Kidder is also a faculty member of the Department of Psychology at The George Washington University, a former teacher in both private and public education and a licensed school psychologist who has worked extensively in public education and private practice helping children with learning disabilities and/or ADHD and their parents. Additionally, she spent almost 15 years working with psychiatrists specializing in the treatment of ADHD and LD in children and adolescents. She was also the national consultant for the General Educational Development Testing Service (GED) where she helped develop a national training program for adult administrators on LD and/or ADHD in adults. She also has presented nationwide on the topic of learning disabilities and ADHD.