What Works in Education –
Introduction to First Edition

Judith Crandall, John Jacobson, & Howard Sloane, Editors

Statement of the Problem

Contemporary trends involving anti-science and postmodern methodologies (e.g., social constructivism) among many academics have fueled resistance to monitoring of individual instructional effects (Bateman, 1995) and antagonism to the scrutiny of the relationships between different instructional procedures and outcomes (Wang, Haertel, & Walberg, 1993a) in the development of better public accountability in local education. This philosophical stance has had a powerful influence on education, especially on educational training institutions. Its proponents exert considerable pressure on educators to adhere to an “educationally correct” model which not only strongly opposes the systematic instruction of specific skills but also the objective evaluation of attainment of these skills. Heward and Cooper (1992) state:

The holistic camp... opposes both systematic instruction of specific skills and objective evaluation... With (the resulting) passive, indirect, non-interventionist instruction, many children will not acquire even minimally functional reading, writing, computing, vocational, and social skills. (pp. 359-361)

There is, however, evidence of support for the use of effective, proven educational strategies in producing improved skills and knowledge. Kagan (1990) states that “educational researchers who argue among themselves about the irrelevance of student achievement merely demonstrate how badly out of touch they are with the public mood of parents, legislators, and district school boards" (p. 458). Hiebert (1994) questions why the instructional components that have been associated with a high level literacy are not a given in all Chapter I programs (p. 24). The problems in special education parallel those found in regular education. Disparagement of the use of performance criteria, or for that matter, objective and uniformly applied criteria to ascertain whether students are learning the skills that schools and communities have specified, is a common and powerful thread running through much of educational policy affecting special education. For example, Palincsar and McPhail (1993) state that “... we are struck by the irony of a knowledge base that is founded on so shaky a foundation as achievement measures, which are widely recognized as at best imperfect and at worst downright pernicious” (p.331).

Another example of the extent to which this has occurred is found in the recommendations contained in the policy agenda from the President’s Committee on Mental Retardation (Nisbett, 1994). The agenda is virtually silent on issues of instructional quality (effectiveness in teaching specific skills and knowledge), instead focusing on the organization and location of educational activities.

Policy analysts have suggested that one reason the focus on specified outcomes for teaching knowledge and skills has been diverted to an emphasis on social outcomes is because the process of defining the outcomes has become politicized. Manno (1995) suggests that consensus is needed on the “academic knowledge, skills, and understandings all children must master if they are to become productive members of society” (p. 23).

The development of valid, fair, and useful performance criteria and measures for educational effects is challenging, but the alternative means and procedures that have been suggested are even more problematic (Dixon & Carnine, 1992; Linn, 1994; MacDonald, 1995; Wang, Haertel & Walberg, 1993a, 1993b). Such approaches are often found to be very labor intensive and many do not generalize effectively across differing educational situations, (Linn, 1994, p. 12).

In the absence of substantive outcome data, attempted improvements in special education have entailed the adoption of promising exemplary or emergent practices that are characterized generally as “best practices” (Peters & Heron, 1993). However, “... inconclusive but encouraging results are often interpreted as best practice when they favor a particular bias” (p. 377) and “... practices often cited by authors as best practice lack sufficient empirical support” (p. 378).

By contrast, there is a significant segment of the special education research community that not only maintains that instructional effectiveness is a concern but also that decision making regarding the implementation of improvements must rest on sound, data-based premises (Manno, 1995). Further, some find attacks on data-based decision making as destructive to the educational process. “Major improvements must be made in the data used to analyze U.S. education if these are to be effectively used to diagnose ills and develop corrective actions... We must move past anecdotal evidence to data-based decision making” (Carson, Huelskamp, & Woodall, 1993, p. 310).

For both special education and regular education students, objective measurement of clearly defined and important academic and life skills is a critical part of their education. It not only provides teachers with information necessary to improve instructional programs, but also prepares students for future, “real world,” on-the-job performance evaluations.

Another reform trend which has characterized the last decade is the call for educators to seek “innovative” solutions. Unfortunately, innovation has come to be associated with the large-scale implementation of untried and unproven educational interventions. The National Center for the Improvement of the Tools of Educators cautions in a handbook for site councils:

Genuine educational reform requires an understanding of the difference between innovation and reform. While innovations result in change, not every innovation in education is a reform. By definition, reform entails improvement; in education, this improvement should be in the form of student learning. (Oregon Education Association/OACE & the National Center to Improve the Tools of Educators, undated)

In the same publication, NCITE advises educational decision makers to judge the potential value of an approach by answering the following questions:

1. Are the approach and its outcomes clearly defined?
2. What evidence exists that the approach is effective?
3. Is an accountability process built into the approach?
4. Is the approach sustainable?
5. Is the approach equitable?
6. Are the costs of the approach and its implementation reasonable?

Imagine the money, time, and energy that would be saved if the adoption of every educational intervention were evaluated and either adopted or rejected using these criteria. What would students and teachers achieve if we systematically implemented programs that we know work?

Programs and Procedures Included in This Publication

Programs and procedures included in What Works in Education share certain things in common. They meet the criteria listed above by NCITE. In addition, they:

• are either based on a series of replicated studies indicating educational effectiveness or they have been used in an experimental setting in which data on effectiveness was gathered,
• routinely gather and compile objective standardized data on student performance on an annual basis,
• routinely gather and compile ongoing criterion-referenced data based on mastery of specified learning objectives,
• use this data to guide decision making about instruction throughout the school year; continuous improvement of instruction is based on collection of objective data and its analysis,
• are replicable, and information is available to the public about where to obtain guidelines for replication,
• closely link teacher accountability, teacher training, and student performance,
• support the implementation and maintenance of “high standards” by specifying learning outcomes, expected levels of performance, and objective measures for determining if these have been met.

It is interesting to note that many of the included programs and procedures were developed as many as 30 years ago and have been revised and improved continuously since their initial development.

The Cambridge Center for Behavioral Studies continues to solicit submissions for inclusion in future volumes of the publication.

References

Bateman, B. D. (1995). Who, how, and where: Special education’s issues in perpetuity. In J. M. Kauffman & D. P. Hallahan (Eds.), The illusion of full inclusion (pp. 75-90). Austin, TX: Pro-Ed.

Carson, C. C., Huelskamp, R. M., & Woodall, T. D. (1993). Perspectives on education in America: An annotated briefing (Sandia National Laboratories). Journal of Educational Research, 89, 259-310.

Dixon, R. C., & Carnine, D. W. (1992). A response to Heshusius’ “Curriculum-based assessment and direct instruction: Critical reflections on fundamental assumptions.” Exceptional Children, 58, 461-463.

Heward, W. L., & Cooper, J. O. (1992). Radical behaviorism: A productive and needed philosophy for education. Journal of Behavioral Education, 2, 345-365.

Hiebert, E. H. (1994). Reading recovery in the United States: What difference does it make to an age cohort? Educational Researcher, 87(21), 15-25.

Kagan, D. M. (1990). Teacher cognition. Review of Educational Research, 60, 419-469.

Linn, R. L. (1994). Performance assessment: Policy promises and technical measurement standards. Educational Researcher, 87(12), 4-14.

MacDonald, H. (1995). Why Johnny can’t write. Public Interest, 120, 3-13.

Nisbet, J. (1994). Education reform: Summary and recommendations. In President’s Committee on Mental Retardation (Ed.), The national reform agenda and people with mental retardation: Putting people first (pp. 151-165). Washington, DC: Author.

Manno, B. V. (1995). Educational outcomes do matter. Public Interest, 119, 19-27.

Oregon Educational Association & National Center for the Improvement of the Tools of Educators. (undated). How site councils help improve teaching & learning: A handbook for site councils and educational leaders on school improvement. Eugene, OR: National Center for the Improvement of the Tools of Educators.

Palincsar, A. S., & McPhail, J. C. (1993). A critique of the metaphor of distillation in "Toward a knowledge base for school learning." Review of Educational Research, 63, 327-334.

Peters, M. T., & Heron, T. E. (1993). When the best is not good enough: An examination of best practice. Journal of Special Education, 26, 371-385.

Wang, M. C., Haertel, G. D., & Walberg, H. J. (1993a). Toward a knowledge base for school learning. Review of Educational Research, 63, 249-294.

Wang, M. C., Haertel, G. D., & Walberg, H. J. (1993b). Toward a knowledge base: Why, how, and for whom? Review of Educational Research, 63, 249-294.

Back to What Works in Education intro page

Return Home