This document provides a set of standards to guide decisions about assessing the teaching and learning of literacy. In the past 30 years, research has produced revolutionary changes in our understanding of language, learning, and the complex literacy demands of our rapidly changing society. The standards proposed in this document are intended to reflect these advances in our understanding.
Readers of this document most likely share common experiences with respect to literacy and assessment. For example, in our own school days, we were directed to read to get the correct meaning of a text so that we could answer questions put to us by someone who already knew that correct meaning or by a test (often multiple choice) for which the correct answers were already determined. In order to develop assessment practices that serve students in an increasingly complex society, we must outgrow the limitations of our own schooling histories and understand language, literacy, and assessment in more complex ways. Literacy involves not just reading and writing, but a wide range of related language activities. It is both more social and more personal than a mere set of skills.
The need to understand language is particularly important. Language is not only the object of assessment but also part of the process of assessment. Consequently, any discussion of literacy assessment must include a discussion of language—what it is, how it is learned, and how it relates to assessment. Before we state our assessment standards, then, we will give an overview of what we mean by assessment and how we understand language and its relationship to assessment.
The Nature of Assessment
The Nature of Language
The Nature of Literacy
The Learning of Language
The Assessment of Language
The Language of Assessment
Using This Document
The Nature of Assessment
For many years, a transmission view of knowledge, curriculum, and assessment dominated and appeared to satisfy our social, political, and economic needs. Knowledge was regarded as a static entity that was “out there” somewhere, so the key educational question was, How do you get it from out there into students’ heads? The corollary assessment question was, What counts as evidence that the knowledge really is in their heads? In a transmission view, it made sense to develop educational standards that specified the content of instruction before developing assessment procedures and engagements.
In the 1920s, notions of the basic purposes of schooling began to shift from an emphasis on the transmission of knowledge to the more complex nurturing of independent and collaborative learning and of problem solving. This shift has gained increasing prominence in today’s postindustrial society, with its ever-expanding need for workers with strong communication skills and dispositions toward problem solving and collaborating. A curriculum committed to independent learning is built on the premise that inquiry, rather than mere transmission of knowledge, is the basis of teaching and learning.
This shift from knowledge transmission to inquiry as a primary goal of schools has important implications for assessment. In a knowledge-?transmission framework, tests of static knowledge can suffice as assessment instruments. Students are the participants who are primarily accountable (either they have the knowledge or they don’t), with teachers held accountable next. Policymakers, including school board members, trustees, or regents, are the primary recipients of assessment data. An inquiry framework changes the role of assessment and the roles of the participants. Within this framework, assessment is the exploration of how the educational environment and the participants in the educational community support the process of students as they learn to become independent and collaborative thinkers and problem solvers. This exploration includes an examination of the environment for teaching and learning, the processes and products of learning, and the degree to which all participants—students, teachers, administrators, parents, and board members—meet their obligation to support inquiry.
Such assessments examine not only learning over time but also the contexts of learning.
Inquiry emphasizes different processes and types of knowledge than does knowledge transmission. For example, it values the ability to recognize problems and to generate multiple and diverse perspectives in trying to solve them. An inquiry stance asserts that while knowledge and language are likely to change over time, the need for learners at all levels (students, teachers, parents, administrators, and policymakers) who can solve new problems, generate new knowledge, and invent new language practices will remain constant. An inquiry perspective promotes problem posing and problem solving as goals for all participants in the educational community. For example, inquiry values the question of how information from different sources can be used to solve a particular problem. It values explorations of how teachers can promote critical thinking for all students. And it raises the question of why our society privileges the knowledge and cultural heritage of some groups over others within current school settings.
Inquiry fits the needs of a multicultural society in which it is essential to value and find strength in cultural diversity. It also honors the commitment to raising questions and generating multiple solutions. Various stakeholders and cultural groups provide different answers and new perspectives on problems. Respecting difference among learners enriches the curriculum and reduces the likelihood of problematic curricular narrowing.
Just as the principle of inquiry values difference, so the principle of difference values conversation over recitation as the primary mode of discourse. In a recitation, it is assumed that one person, the teacher, possesses the answers and that the others, the students, interact with the teacher and one another in an attempt to uncover the teacher’s knowledge. In a conversation, all of the stakeholders in the educational environment (students, parents, teachers, specialists, administrators, and policymakers) have a voice at the table as curriculum, standards, and assessments are negotiated. Neither inquiry nor learning is viewed as the exclusive domain of students and teachers; both are primary concerns for all members of the school community. For example, administrators ask themselves hard questions about whether the structures they have established support staff development, teacher reflection, and student learning. School board members ask themselves whether they have lived up to the standards they have set for themselves and their schools to provide teachers and students with the resources they need to guarantee learning opportunities.
Quality assessment, then, hinges on the process of setting up conditions so that the classroom, the school, and the community become centers of inquiry where students, teachers, and other members of the school community investigate their own learning, both individually and collaboratively. The onus of assessment does not fall disproportionately upon students and teachers (which is often the case in schools today); instead, all those inquiring into the nature and effectiveness of educational practices are responsible for investigating the roles they have played. Different members of the school community have different but interacting interests, roles, and responsibilities, and assessment is the medium that allows all to explore what they have learned and whether they have met their responsibilities to the school community.
The Nature of Language
Language is very much like a living organism. It cannot be put together from parts like a machine, and it is constantly changing. Like a living organism, it exists only in interaction with others, in a social interdependence. Language is a system of signs through and within which we represent and make sense of the world and of ourselves. Language does not contain meaning; rather, meaning is constructed in the social relationships within which language is used. Individuals make sense of language within their social relationships, their personal histories, and their collective memory. In order to make sense of even a single word, people take into account the situation and their relationship with the speaker or writer.
Take, for example, family, a word often used as if all members of society agree on its meaning. The word may mean different things in different contexts, however, whether cultural, situational, or personal. To a middle-aged white person whose parents moved across country with their two children and who repeated that experience herself, family may mean the nuclear family structure in which she grew up and in which she is raising her own children. To someone from a different culture—perhaps an African American or Asian American—the word may conjure images of the constellation of grandparents, aunts, uncles, and cousins who live together or near one another. So, meaning may vary from one person to another, as in this case, where meanings attached to the word family are likely to differ depending on one’s own experience in the family or families one has lived with. Thus, individuals make different sense of apparently similar language to the extent that their cultural and personal histories do not coincide. Consequently, when we attempt to standardize a test (by making it the same for everyone), we make the tenuous assumption that students will all make the same meaning from the language of our instructions and the language of the individual items.
Different cultures also have different ways of representing the world, themselves, and their intentions with language. For example, in any given cultural group, people have different ways of greeting one another, depending on the situation (e.g., a business meeting, a funeral, a date) and on their relationship to each other. Our own language practices come from our cultural experience, but they are also part of the collective practice that forms the culture. Indeed, the different ways people use language to make sense of the world and of their lives are the major distinguishing features of different cultural groups.
At the same time, language is always changing as we use it. Words acquire different meanings, and new language structures and uses appear as people stretch and pull the language to make new meanings. Consequently, the meaning that individuals make from language varies across time, social situation, personal perspective, and cultural group.
The Nature of Literacy
The nature of literacy is also continually changing. Today, many children read more online than offline. They are growing into a digital world in which relatively little reading and writing involves paper, most reading and writing involves images as much as print, and writing (both formal and less formal, the latter including e-mail, texts, Facebook posts, etc.) is becoming equal to, or even supplanting, reading as a primary literacy engagement. The tools of literacy are changing rapidly as new forms of Internet communication technology (ICT) are created, including (at the time of writing) bulletin boards, Web editors, blogs, virtual worlds, and social networking sites such as Ning and MySpace. The social practices of literacy also change as a result of using digital technologies, as does the development of language. New literate practices are learned and refined just by existing from day to day in what has become known as the mediasphere. For example, living with cell phones leads to texting, which changes how people view writing and how they write, and frequenting Web 2.0 sites, such as the video-sharing service YouTube, privileges a visual mode and shapes both attention to and facility with other modes of meaning making. The literacies children encounter by the end of their schooling were unimagined when they began.
Reading and writing online changes what it means to read, write, and comprehend. Literacy practices now involve both the creation and use of multimodal texts (broadly defined). Creating multimodal texts requires knowing the properties and limitations of different digital tools so that decisions can be made about how best to serve one’s intentions. Participating in social networking sites, for example, requires new literacy practices; new literacy practices shape how users are perceived and how they construct identities. This leads to new areas needing to be assessed, including how youths create and enhance multiple identities using digital tools and virtual spaces. We now need to be concerned with teaching and assessing how students take an idea in print and represent it with video clips for other audiences. Similarly, we must be concerned about the stances and practices involved in taking an idea presented in one modality (e.g., print) and transcribing or transmediating it into another (e.g., digital video), and we must consider what possibilities and limitations a particular mode offers and how that relates to its desirability over other modes for particular purposes and situations. Children use different comprehension strategies online and offline, and assessments of the two show different pictures of their literacy development. Online readers, by choosing hypertext and intertext links, actually construct the texts that they read as well as the meanings they make. New multimodal texts require new critical media literacies, linked to classical critical literacy notions of how media culture is created, appropriated, and subsequently colonizes the broader notions of culture—for example, how youth culture is defined by and used to define what youths do, what they buy, and with whom they associate.
The definitions of literacy that have dominated schooling and are insisted on by most current testing systems are inadequate for a new, highly networked information age. Failure to help all students acquire literacies for this age will not serve them or society well. Not to teach the necessary skills, strategies, dispositions, and social practices is to deny children full access to economic, social, and political participation in the new global society. Not to assess these capabilities will result in curricular neglect and a lack of information to inform instruction.
The Learning of Language
By the time children arrive at school, they have learned to speak at least one language and have mastered most of the language structures they will ever use. Through social interaction, using the language they hear around them from birth, they have developed, without their awareness, the underlying rules of grammar and the vocabulary that give meaning to the world as they see it. Nonetheless, we often teach language in schools as if children came to our classrooms with little or no language competence. Nothing could be further from the truth. Children can request, demand, explain, recount, persuade, and express opinions. They bring to school the ability to narrate their own life histories. They are authors creating meaning with language long before they arrive at school.
As children acquire language in social interaction, particularly with others whose language is different or more complex, they gain flexibility in using language for different purposes and in different social situations. Learning a second language or dialect roughly parallels learning the first, for learning any language also entails becoming competent in the social relationships that underlie it. Children also develop fluent use of language without explicit knowledge of or instruction in rules and grammars. This means that grammars and rules are taught most productively as tools for analyzing language after it has been acquired. Even adults who have considerable facility with the language frequently can articulate few, if any, grammar or language rules. In spite of this truism, we often go about assessment and instruction in schools as if this were not the case.
Furthermore, although we pretend otherwise, language is not acquired in any simple hierarchical sequence.
In some ways, school actually plays a modest role in language acquisition, the bulk of which occurs outside of school. In schools, we must learn to teach language in a way that preserves and respects individuality at the same time that we empower students to learn how to be responsible and responsive members of learning communities. In other words, we must respect their right to their own interpretations of language, including the texts they read and hear, but we must help them learn that meaning is negotiated with other members of the learning communities within which they live and work. To participate in that negotiation, they must understand and be able to master the language practices and means of negotiation of the cultures within which they live. They must understand the language conventions that are sanctioned in different social situations and the consequences of adhering to or violating those conventions.
Although much of our language is learned outside school, studying language is the foundation of all schooling, not just of the language arts. For example, in science class, we make knowledge of the world using language. To study science, then, we must study the language through which we make scientific knowledge, language that has an important impact on the curriculum. If in reading and writing about science the language is dispassionate and distancing, then that is part of the knowledge that students construct about science, part of the way they relate to the world through science.
The Assessment of Language
Our description of language and language learning has important implications for the assessment of language, first because it is the object of assessment (the thing being assessed) and second because it is the medium of assessment (the means through and within which we assess). Instructional outcomes in the language arts and assessment policies and practices should reflect what we know about language and its acquisition. For example, to base a test on the assumption that there is a single correct way to write a persuasive essay is a dubious practice. Persuading someone to buy a house is not the same as persuading someone to go on a date. Persuading someone in a less powerful position is not the same as persuading someone in a more powerful position—which is to say that persuasive practices differ across situations, purposes, and cultural groups. Similarly, that texts can (and should) be read from different perspectives must be taken as a certainty—a goal of schooling not to be disrupted by assessment practices that pretend otherwise. To assert through a multiple-choice test that a piece of text has only one meaning is unacceptable, given what we know of language.
Moreover, to the extent that assessment practices legitimize only the meanings and language practices of particular cultural groups, these practices are acts of cultural oppression. When our assessments give greater status to one kind of writing over another—for example, expository writing over narrative writing—we are making very powerful controlling statements about the legitimacy of particular ways of representing the world. These statements tend to be reflected in classroom practices.
When we attempt to document students’ language development, we are partly involved in producing that development. For example, if we decide that certain skills are “basic” and some are “higher level,” and that the former need to be acquired before the latter, that decision affects the way we organize classrooms, plan our teaching, group students, and discuss reading and writing with them. The way we teach literacy, the way we sequence lessons, the way we group students, even the way we physically arrange the classroom all have an impact on their learning.
The Language of Assessment
Because it involves language, assessment is an interpretive process. Just as we construct meanings for texts that we read and write, so do we construct “readings” or interpretations of our students based upon the many “texts” they provide for us. These assessment texts come in the form of the pieces that students write, their responses to literature, the various assignments and projects they complete, the contributions they make to discussions, their behavior in different settings, the questions they ask in the classroom or in conferences, their performances or demonstrations involving language use, and tests of their language competence. Two different people assessing a student’s reading or writing, his or her literate development, may use different words to describe it.
In classrooms, teachers assess students’ writing and reading and make evaluative comments about writers whose work is read. The language of this classroom assessment becomes the language of the literate classroom community and thus becomes the language through which students evaluate their own reading and writing. If the language of classroom assessment implies that there are several interpretations of any particular text, students will come to gain confidence as they assess their own interpretations and will value diversity in the classroom. If, on the other hand, the language of classroom assessment implies that reading and writing can be reduced to a simple continuum of quality, students will assess their own literacy only in terms of their place on that continuum relative to other students, without reflecting productively on their own reading and writing practices.
When teachers write report cards, they are faced with difficult language decisions. They must find words to represent a student’s literate development in all its complexity, often within severe time, space, and format constraints. They must also accomplish this within the diverse relationships and cultural backgrounds among the parents, students, and administrators who might read the reports. Some teachers are faced with reducing extensive and complex knowledge about each student’s development to a single word or letter. This situation confronts them with very difficult ethical dilemmas. Indeed, the greater the knowledge the teacher has of the student’s literacy, the more difficult this task becomes.
But it is not just classroom assessment that is interpretive. The public “reads” students, teachers, and schools from the data that are provided. Parents make sense of a test score or a report card grade or comment based on their own schooling history, beliefs, and values. A test score may look “scientific” and “objective,” but it too must be interpreted, which is always a subjective and value-laden process.
The terms with which people discuss students’ literacy development have also changed over time. For example, in recent history, students considered to be having difficulty becoming literate have acquired different labels, such as basic writer, remedial reader, disadvantaged, learning disabled, underachiever, struggling student, or retarded reader. These different terms can have quite different consequences. Students described as “learning disabled” are often treated and taught quite differently from students who are similarly literate but described as “remedial readers.”
Further, assessment itself is the object of much discussion, and the language of that discussion is also important. For example, teachers’ observations are often described as informal and subjective and contrasted with test results that are considered “formal” and “objective.” The knowledge constructed in a discussion that uses these terms would be quite different from that constructed in a discussion in which teachers’ observations were described as “direct documentation” and test results as “indirect estimation.”
Assessment terms change as different groups appropriate them for different purposes and as situations change. Recent discussions about assessment have changed some of the ways in which previously reasonably predictable words are used, belying the simplicity of the glossary we include at the end of this document. For example, the term norm-referenced once meant that assessment data on one student, typically test data, were interpreted in comparison with the data on other students who were considered similar. A norm-referenced interpretation of a student’s writing might assert that it is “as good as that of 20 percent of the students that age in the country.” Similarly, the term criterion-referenced assessment once meant simply that a student’s performance was interpreted with respect to a particular level of performance—either it met the criterion or it did not. Recently, however, it has become much less clear how these terms are being used. The line between criterion and norm has broken down. For example, criterion has recently come to mean “dimension” or “valued characteristic.” Norm has come to be used in much the same sense. But even in the earlier (and still more common) meaning, most criteria for criterion-referenced tests are arrived at by finding out how a group of students performs on the test and then setting criteria in accord with what seems a reasonable point for a student’s passing or failing the test.
In other words, assessment is never merely a technical process. Assessment is always representational and interpretive because it involves representing children’s development. Assessment practices shape the ways we see children, how they see themselves, and how they engage in future learning. Assessment is social and, because of its consequences, political. As with other such socially consequential practices, it is necessary to have standards against which practitioners can judge the responsibility of their practices.
Using This Document
In what follows, each standard is presented as a statement with a brief explanatory paragraph. The standard is then expanded with additional detail. The text concludes with case studies that illustrate the standards’ implications in both large-scale and classroom assessments.
The central premise of the standards is that quality assessment is a process of inquiry. It requires gathering information and setting conditions so that the classroom, the school, and the community become centers of inquiry where students, teachers, and other members of the school community examine, individually and collaboratively, their learning and ways to improve their practice.
Case Studies 1 & 2
Case Studies 3 & 4
NCTE-IRA Joint Task Force on Assessment