National Council of Teachers of English Logo

An Analysis of the NCTQ/U.S. News & World Report Plan to Evaluate and Rank Teacher Education Programs in Colleges and Universities

In January, 2011, the National Council on Teacher Quality (NCTQ) and U.S. News & World Report announced their intention to evaluate and rank teacher education programs in the United States. The Conference on English Education (CEE), a U.S.-based international organization of English education professionals affiliated with the National Council of Teachers of English, is providing this response to the NCTQ/U.S. News & World Report plan as a service to CEE members, institutions of high education, policy makers, and the general public.

The Conference on English Education, has approximately 1400 members, the majority of whom are engaged in the education of English/Language Arts teachers at the undergraduate and graduate levels. Most work in departments, schools, and colleges of education, or in departments of English in colleges of arts and sciences.

Before providing our response to the NCTQ recommendations, CEE stresses that its members respect all efforts to improve teacher education in U.S. universities. Although we do not believe teacher education programs merit the hostility that sometimes accompanies critiques of their work, we understand that any educational effort can be improved. At the same time, we believe that efforts to evaluate various aspects of education have not always been held to sufficiently high standards of rigor and validity. In that spirit, CEE offers this response to the NCTQ National Review of Education Schools in hopes that it and other external critiques of teacher education will be improved and thus become more credible and useful to those whose work is the subject of such evaluations.

  1. The Development of the NCTQ Standards for Rating the Nation's Education Programs
    NCTQ has put forth a set of standards that in many instances are not research based and that were not submitted to an inclusive or transparent process of development and review. We suggest the standards be rethought using a process similar to that used in the recent development of the Common Core State Standards. For those standards, a roster of development team members was made public, early drafts were placed on a public web site, and input was solicited from all stakeholders—teachers, administrators, parents, the general public. Comments resulted in substantive changes.  A similar process has typically been used in the development of content standards at the state level. However, in the case of the NCTQ Standards, the names and qualifications of the development team were not published, stakeholders were not offered the opportunity to provide input on the standards, and no school principals or classroom teachers—those who ultimately work with the graduates of teacher education programs—were members of the NCTQ Technical Panel, a group selected by NCTQ that conducted the only review of the NCTQ standards. A process similar to the one used to develop the Common Core State Standards would help insure that the standards are useful to stakeholders and broadly accepted by the general public.

    Stronger rationales are needed for the NCTQ standards. The standards are often simply asserted, without an adequate explanation of why the standard is essential or valid. Throughout the rationales, there are statements such as “While there is no research evidence . . . it has been posited that,” “This standard begins with a common-sense presumption,” and “While the evidence . . . is sparse .” Such constructions show the need for citation. In many cases there is research evidence to counter claims asserted in the NCTQ standards, yet such evidence-based opposing claims are never entered in dialogue with those preferred by the standards developers. Further, when NCTQ claims are “posited” by means of passive voice constructions (e.g., “it has been posited that”), those whose work will be subject to NCTQ evaluation need to know the source for the claims and the evidence on which it is based. When such vague and unsubstantiated claims are made, the reader has no assurance that the standards have a firm foundation in research or have benefited from a systematic process of consultation with the profession.

    Thus, the NCTQ Standards appear to simply represent one group's beliefs about how teachers should be prepared. Another group might have different beliefs. For instance, many teachers, administrators, and parents agree that prospective teachers should know something about child and adolescent development to teach well. However, the NCTQ standards are silent on this matter. Similarly, many teachers, administrators, and parents agree that prospective secondary school teachers should know something about working with English language learners. However, the NCTQ standards mention coursework focusing on ELL learners only for elementary teachers.

    The reason to engage members of the profession in a standards-setting and evaluation project is that their support will be needed if substantive changes are to be made to programs and policies. In our judgment, the inclusion of a wide and diverse range of teacher educators in the NCTQ development process would result in higher quality standards and a greater likelihood that the standards will be respected, embraced, and put to use.
      
  2. Syllabi and Other Program Materials as a Source of Evaluation Data
    According to the NCTQ/U.S. News & World Report plan, NCTQ will determine whether or not programs meet standards by analyzing syllabi, course and program descriptions, and instruments used to evaluate students. Previous critiques of NCTQ from groups such as the Education Deans, Directors, and Presidents from AAU Universities have argued that these limited materials will be unlikely to provide an accurate portrait of particular programs or, more importantly, an accurate measure of the quality of a program's graduates. We urge NCTQ to listen to these critiques.

    To show that syllabi and program materials can provide adequate information to conduct a valid evaluation, NCTQ offers this analogy (in the “FAQ” document on their website): “If a syllabus for an early American history course contains no mention of topics associated with the American Revolution, one might rightfully suspect that the course is deficient, because the Revolution is considered a basic, essential topic.” The implication of this analogy is that NCTQ’s standards and indicators focus on such “basic, essential” elements of teacher education that syllabi, course descriptions, and other materials will be sufficient to show that standards have been met. Unfortunately, many of the indicators NCTQ intends to use to score the syllabi and other materials do not pass their own “American Revolution” test: 

    For example, on Standard 1: Classroom Management, Indicator 1.5 requires that programs

    Allow the cooperating teacher to document his/her evaluation of the candidate’s classroom management techniques in one of the following ways: Using the same evaluation instrument used by the student teacher’s supervisor; Using an evaluation instrument that is substantially similar to that used by the student teacher’s supervisor; [or] Recording his or her evaluation on the student teacher’s supervisor’s evaluation instrument.

    The emphasis appears to be on giving the cooperating teacher a chance to provide written feedback, and on using an instrument similar to the one used by the supervisor. However, it is quite possible to imagine programs wanting cooperating teachers to record their evaluation on an instrument different from the one used by the supervisor. For instance, they might do so to provide a different lens through which the candidate’s performance can be viewed. We would agree that having a feedback system in student teaching is essential. However, insisting that program materials show this particular kind of feedback system is comparable to insisting that a syllabus for an American history course show that the American Revolution is being taught in a particular way.

    Similarly, in Standard 5: Student Teaching, Indicator 5.4 says, “The program does not allow teacher candidates to take any course other than a companion seminar during student teaching.” Again, a program that allows candidates to take more than one course during student teaching is not difficult to imagine. Perhaps programs do so because they have found certain courses help candidates better understand particular aspects of student teaching if taken during student teaching. However, such rationales behind decisions about program structure are often not part of syllabi, course descriptions, or program descriptions. It seems unreasonable to suggest that a program that is silent on this matter of program structure is comparable to an American history course that is silent about substantive content issues such as the American Revolution.

    In short the standards and indicators too often seem incompatible with the nature of syllabi and other pre-existing program materials on which the NCTQ evaluation will be based.
       
  3. The Process of Analysis
    As stated above, a program's final score/rank in the NCTQ evaluation will be determined by the extent to which indicators for each standard are present in the syllabi and other materials the institution provides. However, the information on NCTQ’s web site says very little about how a given piece of evidence will be analyzed to determine whether or not a standard is met. We suggest NCTQ provide information about how the materials will be analyzed.

    Furthermore, in addition to being unsupported by research, the indicators themselves, which are the key element in the process of analysis, are often unclear or poorly defined. For instance, among the indicators for Standard 1:  Classroom Management is the following:

    [The instruments used to evaluate student teachers] specifically address the student teacher's appropriate use of low profile desists for managing minimally disruptive behavior.

    A search for the term "low profile desists" in multiple commonly-used databases—JSTOR, PsychInfo, Ed Research Complete—produces no articles or studies that make use of the term. A search in Google Books produces a single result, in a book titled Strategies for Effective Teaching (coauthored by NCTQ Advisory Board member Thomas Lasley). Given that the concept of “low profile desists” must be "specifically addressed” by programs, and given that the term “low profile desists” is virtually absent in scholarly literature and thus unlikely to appear on a course syllabus, NCTQ readers will have to read between the lines of the syllabi or other materials to determine if the concept is present in a course. The requirement that syllabi show such an obscure classroom management procedure might easily have been avoided by asking teacher educators in the field to provide feedback before the system was put in place.

    Similarly, Standard 2: Practice Planning Instruction includes Indicator 2.6, which states that “None of the program's instructional planning assignments encourage [sic] candidates to use pseudo-scientific methods of instruction.” The term “pseudo-scientific” is neither defined nor discussed. What will determine whether something on a syllabus is “pseudo-scientific”? Is there a list of methods NCTQ considers pseudo-scientific, or will expert judgment come into play on a case-by-case basis? In either case, what definition of “science” will be used to ensure that “pseudo-science” is identified in valid and reliable ways? Given that the NCTQ standards themselves are, for the most part, not justified by scientific research but by “common-sense presumption[s], ” “consultations with expert panels,” and simple assertion, what standards will be used to determine what counts as “pseudo-science”?

    Taken together, the standards and indicators present a narrow, and, in some instances, arbitrary view of teacher education.  Furthermore, the language used to describe the indicators is not clear enough to give those outside NCTQ confidence that they will be applied in a fair and reasoned manner.  The problems that can be caused by this lack of clarity were apparent in NCTQ’s 2010 reviews of Illinois and Texas teacher education programs. Penelope Peterson, Dean of Education at Northwestern University (the only Illinois institutions to receive a grade of A from NCTQ), has said, "Basically, we would have failed for the standards if we hadn't presented additional data and argued with them” (Daily Northwestern, February 17, 2011). We would hope that NCTQ would make revisions that would make it unnecessary for a dean to present additional data and “argue” their way from failure to an A.
       
  4. Alternative Licensure Programs
    In the NCTQ report on Illinois teacher education, alternative providers of teacher preparation are given “narrative evaluations but no ratings” because of their “unique designs.” And now, in the list of institutions to be evaluated for the nationwide report, alternative providers not based at colleges or universities have been excluded entirely. Given NCTQ's stated goal of conducting the first national review of teacher preparation programs and given the very large (and growing) number of teachers being prepared by Teach For America, iteach TEXAS, and other such providers, this exclusion is difficult to understand. Shining a light on traditional programs while allowing others to avoid such scrutiny gives the impression that some programs are being protected or held to different standards, whether that is the intent or not.

Our review of the NCTQ plan for evaluating and ranking teacher education programs has led us to conclude that the grades/rankings given by NCTQ through this process are unlikely to be valid representations of program quality or of the quality of program graduates unless NCTQ makes substantive changes to the standards, indicators, and evaluation procedures. When the NCTQ/U.S. News & World Report project is complete, CEE intends to review the full report and produce an assessment of its quality and findings.

The Conference on English Education welcomes the effort to improve the quality of teachers, which is the end toward which we have dedicated our careers. An evaluation system that teacher educators would endorse would be a great contribution to those who prepare teacher candidates for the complex and challenging work that awaits them in a shifting demographic and policy landscape. Such a system must go beyond what is stated on course syllabi and program descriptions and look more widely and deeply at the factors necessary for success in 21st century classrooms.

Those who wrote and disseminated this response to the NCTQ standards were not paid to do so, were not supported by grant funds, and did not participate in this project at the request of their institutions. Rather, they did so as committed professionals who believe deeply in the importance of excellent teacher education. We hope this response results in improvements to the NCTQ plan so it will be regarded as a rigorous evaluation that provides useful information. In addition, we hope NCTQ finds more systematic and collegial ways to include the teacher education community in its future efforts.

Response Authors:
Sarah Freedman
Suzanne Miller
Peter Smagorinsky
Joyce Stallworth
Don Zancanella

Document and Site Resources

Share This On:

Page Tools:

Join NCTE Today

Copyright

Copyright © 1998-2014 National Council of Teachers of English. All rights reserved in all media.

1111 W. Kenyon Road, Urbana, Illinois 61801-1096 Phone: 217-328-3870 or 877-369-6283

Looking for information? Browse our FAQs, tour our sitemap and store sitemap, or contact NCTE

Read our Privacy Policy Statement and Links Policy. Use of this site signifies your agreement to the Terms of Use

Visit us on:
Facebook Twitter Linked In Pinterest Instagram