ABSTRACT

In response to a growing body of literature in writing studies and education that sounds a call for new models of assessment for today’s digital texts, this chapter offers an adaptable model for the assessment of new media composition in the writing classroom. I begin with a brief survey of literature that provides a rationale and lays a foundation for the assessment model, and then present the model, which draws specifically from three assessment frameworks espoused by Paul Allison (2009), Eve Bearne (2009), and Michael Neal (2011). The model places value on both composition process and product, requires students to be involved in assessment through setting their own goals and participating in guided reflection, and offers students adaptable language to describe new media composition process and product. The chapter concludes with several videos that offer student perspectives on using the model in the writing classroom to assess digital work. The student videos, along with my reflective analysis, evaluate the implementation of the assessment model.

I was dreading this moment. It might seem that after so much work—planning, drafting, conferencing, sharing, revising—after all this, I should have been a bit more excited to interact with and assess the final drafts of my students’ compositions. They had worked hard, and the compositions were creative and innovative. But when the students turned in final drafts, my excitement gave way to apprehension, and I was not really looking forward to the moment when I had to give comments and a grade, shying away for as long as possible from the virtual “stack” of compositions waiting for me on my laptop.

I wasn’t even exactly sure why I was reluctant, because it wasn’t as if I hadn’t thought about new media assessment and the challenges it presents until this final moment: As my students composed drafts of their compositions, I asked them to construct a new media assessment rubric that I planned to use to give feedback and assign grades. The rubric was composed collaboratively, had been discussed in small and large groups, and was intended to help me and my students become aware of and articulate specific qualities that made new media compositions most effective for a chosen audience and purpose. We put this new media rubric together by starting with the rubric we had composed as a class for print compositions, and, as a result, we used much of the same language. For example, the rubric had an “Argument” section that emphasized using a complex claim supported by evidence and rhetorical appeals to logos, pathos, and ethos, and an “Organization” section that required students to have an effective introduction, conclusion, and “clear and coherent” linking between ideas. An “Interest and Style” section stated that students should write in engaging, interesting, and mechanically correct ways, and “Modes and Media” encouraged the effective use of visuals and sounds along with the written word. And, yet, I uneasily approached the assessment of my students’ new media compositions armed with this rubric, hoping—with fingers crossed—that it would guide me in shaping summative feedback in response to the products students had composed as well as formative feedback that would help these students develop as writers.

Foreshadowed by my apprehension, as I began to look through the compositions and give feedback, questions arose. One student’s video on the benefits and drawbacks of Facebook didn’t use “evidence” in the form of statistics or facts, but instead used a series of images and various song lyrics to persuade the audience. Was this evidence specific? Another student turned in an anti-rape visual argument that used stenciled drawings, brilliant colors, and allusive imagery. But where was the introduction and the conclusion in this composition, and was it organized? The rubric validated an author who “made effective choices about size, clarity, organization, color scheme, font, and positioning” of images and effective choices about “volume, timing, transitions, lyrics, and instrumentation” of sounds. But what, specifically, did these “effective choices” look like? Faced with vague and infinite answers to questions like these and an inability to reconcile the language of print writing with the multimodal compositions in front of me, I found our class rubric helpful at times, but ineffective at others. I also found myself struggling with how much feedback to give on the products themselves versus how much to give about composition process, which wasn’t reflected in the final drafts at all. I knew there had to be a better way to assess student new media compositions, but I was floundering.

A “KAIROTIC JUNCTURE”

This moment, filled with dread and confusion, is representative of larger questions for composition studies at both the classroom and programmatic levels: What is valued in digital and multimodal texts? How is this value assessed? And what is the purpose of such assessments? Michael Neal (2011) argued,

with the increasing viability and visibility of digital texts and technologies, we are in what I believe is a finite, kairotic juncture with writing assessment. We have this opportunity, while the texts and technologies are relatively new, to reframe our approaches to writing assessment so that they promote a rich and robust understanding of language and literacy. (p. 5)

My experiences in the classroom demonstrate the truth of Neal’s claim: composition studies has reached a juncture where both programmatic and instructor-oriented approaches to assessment can be reframed in ways that will highlight complex, research-supported understandings of what students can do in the classroom with both printed and digital texts. Others have been calling for new small- and large-scale assessment models for years, suggesting that the responsibility for developing, piloting, and refining assessment tools falls on teachers and researchers of composition, and that now is the time to develop and test new forms of assessment (Huot, 2009; Penrod, 2005; Rea & White, 1999; Sorapure, 2006; Takayoshi, 1996; Whithaus, 2005; Zoetewey & Staggers, 2003).

Compositionists at all levels, however, have been slow to answer this call. This is due, in part, to the complex nature of new media texts and their convergence with theories of writing assessment. The opportunity to reframe approaches to writing assessment is not a simple one, and charging ahead blindly does not seem wise for ourselves or for students. In the field, there is currently no agreed-upon language or vocabulary for discussing new media texts. New composition genres develop and will continue to be in flux, and the tools and technologies used to create them will continue to evolve and change. There isn’t unified agreement, either, as to which qualities traditionally valued in academic print writing can or should carry over to new media texts. With so much up in the air and yet to be theorized, seizing the current kairotic moment to reframe writing assessment for new media can be a daunting and confusing task.

In this chapter, I seek to answer Neal’s (2011) call to reframe writing assessment by beginning where I spend a lot of my time—in the classroom. To do so, I offer a rhetorically sensitive, adaptable model for the assessment of new media composition in a writing course, along with my own and student reflective analyses about its implementation and continuous evolution. The model builds from, synthesizes, and transforms new media assessment frameworks put forth by others, using a form that in itself reflects the rhetorical situation in which it is designed to function. I begin the chapter by briefly surveying work done in writing studies and education that provides a rationale and lays a foundation for the model. Drawing more specifically from new media assessment frameworks espoused by Paul Allison (2009), Eve Bearne (2009), and Michael Neal (2011), I have constructed the model to place value on both composition process and product, to require students to be involved in continuous self-assessment through setting their own goals and participating in guided reflection, and to offer students adaptable language to describe new media composition process and product. In offering this model, I do not expect or recommend that others pick it up and use it as is. Instead, I hope that other instructors and scholars in the field can use the model to open further discussion about what is happening in our theories of composition and in our classrooms at the intersection of writing assessment and new media and to reframe and remediate their own assessment practices within their local contexts. I conclude the chapter with a series of short videos that offer student perspectives on using the model in the writing classroom along with my reflective analysis of how the model works in the classroom.

THE LANDSCAPE OF NEW MEDIA ASSESSMENT

After several semesters of assessing new media compositions using a largely print-centric, product-oriented rubric, I knew I needed to develop a different approach, and so I turned to literature in writing studies and education to see what others were saying about assessing new media texts. First, I realized that a reframing of classroom writing assessment for new media must ground itself in a solid theory of writing assessment itself. Brian Huot (2009) developed a rhetorically sensitive theory of writing assessment and set of preliminary principles and procedures that, although not designed specifically with new media texts in mind, establish the necessity of assessment practices that are rhetorical in nature and sensitive to purpose, audience, and context. Huot’s (2009) five principles for a new theory and practice of writing assessment include that the assessment(s) be site-based, locally controlled, context-sensitive, rhetorically based, and accessible, focused not on “the ability to judge accurately a piece of writing or a particular writer but to be able to describe the promise and limitations of a writer working within a particular rhetorical and linguistic context” (p. 173). And while Huot (2009) established these principles as a first step in constructing a “new theoretical umbrella” for writing assessment, he also urged compositionists to “develop procedures with which to document and validate such assessment” at local levels (p. 171).

Huot’s (2009) site-based and locally controlled principles can be applied in a writing classroom through a focus on the needs, resources, and concerns of students and instructor and through involving students in criteria development and the administration of assessments. Context-sensitive assessment at the classroom level requires alignment between goals set by the instructor and those important to students, and rhetorically based assessment involves criteria that consider first the audience and purpose of the assessment and take form and shape based on this consideration. Accessible assessment in the classroom could mean that students have access to criteria and assessments themselves throughout the process. Huot’s theory reminds me that the classroom-level assessments for particular assignments can and should be focused on specific, local, and instructional needs and monitored and controlled by those who have a stake in them—namely, instructors and students.

New media theorists also offer important concepts and vocabulary for the development of new media assessments that are rhetorical in nature. New media itself can be confusing as a term, and if used, should be defined carefully. My definition of new media is not necessarily tied only to materials, but instead to materials along with the ways they are used in context. Lisa Gitelman (2006) included in her definition “technological forms and their associated protocols” (p. 7), both materials and the social context of production and consumption. Henry Jenkins (2008) picked up Gitelman’s (2006) dual definition, describing media as cultural systems distinct from technologies that function solely as delivery systems. Such a consideration of social protocols and cultural systems within a definition of media is what Anne Frances Wysocki (2004) labeled the materialities of new media. Wysocki emphasized that new media composers keep in mind these materialities—the contexts in which they create texts and the specific purposes and audiences:

We should call “new media texts” those that have been made by composers who are aware of the range of materialities of texts and who then highlight the materiality: such composers design texts that help readers/consumers/viewers stay alert to how any text—like its composers and readers—doesn’t function independently of how it is made and in what contexts. (p. 15)

Thus authors of new media texts, according to Wysocki, do consider materials—tools, words, images, sounds, technologies—but just as importantly, they also consider materialities—the how and the why of composing related to context.

Under this definition, new media texts do not necessarily have to be digital. Any text in which the author keeps its materiality—how it is made, in what contexts, and for what purposes—at the forefront during its creation is a new media text. Wysocki (2004) would not label all electronic texts “new media” either; she stated that new technologies themselves do not “cause us to produce texts that break away from or ask us to think and act differently than print technologies” (p. 19). New media texts, as laid out here, require a shift in how the author thinks and acts during their creation. Wysocki also pointed out that texts that simply combine words, sound, and graphics do not necessarily count as “new media.” Instead, Wysocki tried to “get at a definition that encourages us to stay alert to how and why we make these combinations of materials, not simply that we do it” (p. 19). This definition of new media places focus both on a product and how it is assembled, but also on the process of composing and why decisions are made based on context and audience, and it is this how and why that my assessment model seeks to foster and highlight.

Other new media theorists point to new and developing logics and shifting relationships between authors and audiences in new media composition environments, along with evolving definitions of terms such as persuasion and the use of postmodern logic that appeals “to bodies and feelings rather than articulated logics” (Wysocki, 2010, p. 94). Wysocki (2010) sought to move beyond what Kathleen Welch (1999) labeled the “relentless logic” of Aristotle and Plato to a new performative, postmodern logos that signifies the associative, dispenses with unity, uses repetitive constructions, and commits to mixing and fragmenting. This postmodern logos is physical and bound to bodies and sensory response; it is written, oral, and visual; it is juxtaposed and foregrounds relationships.

Geoffrey Sirc (2010) called this kind of writing “serial composition of short, staccato bursts,” which consists of “well-chosen bricks of meaning combining to form a rich whole,” that use cumulative logic and juxtaposition as communicative strategies (pp. 70–71). Serial composing challenges traditional notions of coherence, and Kathleen Blake Yancey (2004) claimed that today’s composition is “made whole by a new kind of coherence” (p. 89), not created through relationships of words to words and words to context, but instead through repetition, arrangement and re-arrangement, linking, and patterns. These relationships between elements are sometimes like the relationships that occur in print compositions (that is, subordinated, analytical, and logical), but just as often, these relationships are juxtaposed, additive, and associative in nature, or they involve some combination of the subordinate, the additive, the analytical, and the associative. Composition studies needs to explore ways to understand these logics, to help students develop and leverage them in their own work, and then to responsibly assess the effectiveness of these logics in the classroom.

Stuart Selber (2004) offered a theoretical framework that instructors can use to begin to design new media instruction and assessments at the classroom level that includes an emphasis on developing logics (as discussed by, Sirc, 2010; Welch, 1999; Wysocki, 2010; Yancey, 2004). Selber redefined computer literacy by calling for attention instead on multiliteracies, where “teachers should emphasize different kinds of computer literacies and help students become skilled at moving among them in strategic ways” (p. 24). He organized his discussion of multiliteracies according to three categories: functional, critical, and rhetorical. Functional literacy is organized by the “computers as tools” metaphor, where students are users of technology with effective computer use as the goal. Critical literacy is organized by the “computers as cultural artifacts” metaphor, and students become questioners of technology who take part in informed critique. Rhetorical literacy uses the “computers as hypertextual media” metaphor and students learn to be producers of technology with reflective praxis as the objective. Selber emphasized that students must learn to wield all three literacies simultaneously, understanding that the metaphors come together in multiple combinations. Selber’s use of the terms functional, critical, and rhetorical literacies can be adapted for use in the classroom, and I have drawn from his framework to shape my assessment model.

Others in writing studies and education have looked specifically at what the assessment of digital texts can and should look like in the classroom. Madeleine Sorapure (2006) called attention to the multimodal products students create, arguing that writing instructors can look at the relations between modes when they assess student digital writing, using language such as metaphor and metonymy to talk about how the modes comes together to make meaning in a product. Others argue that rubrics used for the assessment of digital writing must draw from established notions of high-quality print writing, but do more than simply transfer these notions and apply them verbatim to digital texts. Instead, criteria must be appropriated and refashioned and new criteria must be added. Meredith Zoetewey and Julie Staggers (2003) pointed to the limitations of what they called “repurposing assessment criteria” from print contexts, and advocated for the transformation of established criteria such as coherence, clarity, and relevance. Diane Penrod (2005) argued that writing instructors now need to “consider different criteria for successful online composing” that measure qualities such as “interactivity, visuality, and aurality combined with writing in a truly authentic context like a web page or a blog” (p. 20). But remediating, transforming, and adding to criteria developed to assess print writing can be tricky. How is a writing instructor to know when a criterion should be discarded entirely or kept and transformed? Sorapure pointed to this same dilemma, asking if criteria such as coherence and clarity actually can transfer from print to digital spaces and whether such criteria need to be revised or even rejected in new media environments. Sorapure encouraged compositionists to attend to differences between the printed and the digital, but at the same time, to work from what we know about good writing instruction.

An alternate approach to assessing new media products in the classroom is to assess instead only composition process. Paul Allison (2009) took this approach, using a detailed “self-assessment matrix” to guide students in composing blog posts, leaving the assessment of the products themselves to the informal evaluations students receive from authentic audiences when the blog posts go live. Penrod (2005) also articulated a process-only approach to assessment, abandoning the terms competence, skill, and product entirely. Instead, “a greater emphasis is placed on communication and community, interactive and multiple discourse situations and formats, and the process of writing in and for networked contexts” (p. 53). For Penrod, this focus on communication, discourse, and process throughout the stages of student electronic writing provides a more authentic assessment, able to function as an exploration of assets and a creative force.

As a midpoint between the two extremes, other teachers of multimedia writing claim that assessment of digital writing (and all writing, really) must take into account both the process students move through to compose texts and the texts themselves as products. Pamela Takayoshi (1996) and Troy Hicks (2009) argued separately that both process and product must be assessed in digital environments. Eve Bearne (2009), as well, suggested that in developing assessment systems within literacy and English/Language Arts, “there needs to be a way of describing the dimensions and characteristics of multimodal texts and, importantly, a way of describing what progress in multimodal reading and text composition might involve” (p. 21). Many instructors use student-authored reflection to achieve both of these goals, a practice that highlights the importance of student participation in the assessment process. Takayoshi claimed that students should become assessors of their own processes and products (p. 253), the National Writing Project (2010) called for “ongoing self-assessment” (p. 91), and Jill McClay and Margaret Mackey (2009), along with Carl Whithaus (2005), advocated for distributed assessment in which both teachers and students participate and through which both parties can develop metacognitive and critical awareness. Neal (2011) added that the most useful student-authored reflection is evidence-based and takes place at multiple points throughout the writing process, not just when it comes time for a grade.

BUILDING THE MODEL:
THREE EXISTING FRAMEWORKS

Allison (2009), Bearne (2009), and Neal (2011) have each put forth frameworks for the assessment of student-authored digital texts—Allison and Neal at the classroom level and Bearne on a larger, curricular scale. I draw from these three frameworks to build my new media assessment model, pulling out aspects of each that I find helpful and that align with the research above, and adding to and transforming other aspects. My hope is that others treat the resulting model in the same manner—extracting what is useful and adapting it according to the needs of assessing new media in specific local contexts.

First, through his self-assessment matrix, Allison (2009) focused his high school students on composition process, asking them to move through and document rigorous stages of writing, research, and design as they composed blog posts. In the matrix, Allison (2009) used four evaluative categories: “Participating (Responding),” “Producing (Drafting),” “Perfecting (Revising and Editing),” and “Publishing,” which offer students and writing teachers a useful approach to valuing and evaluating a multi-part, complex composition process that includes pre-writing, multiple drafts, multimodal expression, critical thinking, collaboration, giving and receiving feedback, and publishing the product. Unlike other composition process assessment models that include only the overly general requirement to “reflect,” Allison’s matrix richly described the various steps students can and should move through to take part in a rigorous composition process that fosters meaning-making and synthesis. Allison’s students were also the principal assessors of the work, as they managed their movement through the matrix and marked off their own progress. The framework does have its limitations, however: It lacks specific assessment of final product along with process, the layout of the matrix might suggest a uniform and linear progression through the composition process, and the use of the term perfecting could place undue emphasis on the appearance of a “flawless” product. Although these three aspects of the matrix could be further developed, Allison’s approach to assessing student work as they compose blog posts recognizes and accounts for student composition processes for digital texts that are complex, layered, and assessed by students themselves as well as by the instructor.

Bearne (2009) also forwarded a framework for multimodal assessment on a curricular scale for K–12 classrooms. The framework integrates attention to composition process through reflection with attention to final product, detailing four areas in which students can show that they are “getting better at multimodal composition”: the selection of mode and content based on specific purpose(s) and audience(s), the structuring of texts, the use of technical features for effect, and reflection. Bearne pointed out that such a framework, implemented on a large scale, could “offer teachers a starting point from which they may develop a vocabulary to talk about multimodal texts” (p. 23). Thus I take up aspects of Bearne’s framework to further develop my vocabulary surrounding new media assessment at the classroom level, and I have incorporated these aspects into my model. First, the framework maintains a focus on the rhetorical principles of purpose and audience, seeking to examine whether choices of mode and content align with the purpose of the text and its intended readers/viewers, a concept that runs throughout the literature on new media composition assessment as well (Herrington, Hodgson, & Moran, 2009; Hicks, 2009; Penrod, 2005; Rea & White, 1999; Sorapure, 2006 ). Bearne’s framework also highlights the function of technical features such as line, color, perspective, sounds, camera angles, punctuation, font choice, and sentence structures. The use of many of these kinds of technical features, while traditionally falling outside the purview of writing teacher expertise, becomes part of the writing classroom when students compose in multiple media beyond the printed word. A focus on how these kinds of technical features align with the purpose and audience of multimedia texts is a helpful addition when thinking about how digital texts do their work and achieve desired effects. Bearne’s “Reflect” category, as well, includes an emphasis on becoming aware of steps in the composition process such as redesigning and redrafting according to purpose and audience, implying that this sort of reflection should take place at multiple points throughout the composition process for a text.

Bearne’s (2009) fourth category suggests that student progress in making multimodal texts is marked by an ability to “Structure texts”—that is, an ability to use structural devices to organize texts, to structure longer texts with “cohesive devices,” and to “integrate and balance modes for design purposes” (p. 22). From my experience, this category does not map as well onto writing classroom new media assessment models such as my own. As the work of scholars such as Geoffrey Sirc (2010), Kathleen Welch (1999), and Wysocki (2010) illustrate, a linear, elaborated logic is often not the predominant or only logic at work in digital spaces. Instead of a cohesive, unified product, some digital texts offer juxtaposed, disparate pieces that composers place alongside or on top of one another for effect. These texts may or may not be organized using sections or screens; however, Bearne’s “Structure texts” category suggests that as students become more experienced with multimodal composing, their texts might adhere to such a linear, cohesive model. The suggestion that skilled student–composers balance modes also excludes the possibility of students making purposeful choices that use only one mode of expression even when multiple modes of expression are available, or multiple modes of expression that aren’t in “balance.” Despite these potential shortcomings, Bearne’s curricular model does provide much that is useful for teachers (like me) who wish to reframe assessment practices for new media, such as an emphasis on the rhetorical elements of audience and purpose and specific attention to technical features.

The final framework I’d like to consider is Neal’s (2011) list of holistic and specific targeted elements that he used, along with evidence-based reflection, as assessment criteria for hypermedia assignments in his college writing classroom. The list of holistic elements includes depth of content; persuasiveness; relationships between modes and media and to traditional conventions; organization; and attention to audience and context. Other specific targeted elements include progression/pace; selection and presentation of images; clarity and strength of written content and audio components; the editing of words, visuals, and technology; and functionality. Neal described how these lists are not comprehensive, but function to guide and frame his feedback. In addition, he comments on the need to couple this feedback with intentional, evidence-based reflection on the students’ part:

In assessing student hypermedia I have rediscovered the value of student reflection. The criteria I typically find the most vital requires students to explain their rationale, critical-thinking, decision-making processes, self assessment, or insights into objectives and goals for me to provide meaningful feedback. (Neal, p. 98)

Neal’s framework builds from course objectives, linking assessment to instruction. Like Bearne (2009), he carried the rhetorical concerns of audience, purpose, and persuasion into his list, which also values careful attention to technical features such as the use of images, written components, audio, and editing. His framework addresses relationships between modes, media, and texts without limiting students.

One place where Neal’s (2011) model could be extended or further developed might be in its use of criteria traditionally linked with print composition such as organization, arrangement, or clarity. These terms may carry different or shifted meanings in new media environments and could be further elaborated for students and for instructors. Additionally, the lists place focus mainly on product, and they might be expanded to take into account composition process in more explicit ways. One of Neal’s key principles for assessing student hypermedia at the classroom level is indeed to “develop criteria for hypermedia that communicate meaningful formative and evaluative feedback” (p. 99), and his framework might be developed using additional criteria that would help instructors give more formative as well as summative feedback.

Perhaps most useful is Neal’s (2011) description of the reflection that partners with teacher feedback based in assessment criteria, and I draw from and build this aspect into my model. He explained that instead of asking students for evaluative judgments on their own work, he instead asks them for “rhetorical rationale for the decisions they made as writers” (Neal, p. 86), asking them questions that often begin with why. This kind of reflection, Neal noted—more useful than “pure self-assessment”—should be performed at regular intervals throughout the writing process, and must be evidence-based.

THE MODEL I PROPOSE

The work of Allison (2009), Bearne (2009), and Neal (2011) provides composition instructors like me with much that is useful when thinking about how to assess new media compositions in the writing classroom. Building on the work of these three scholars, I join them in answering the call to develop new assessment frameworks, and I offer my own model for new media assessment (see Figure 1). The model uses and transforms elements from Allison’s (2009), Bearne’s (2009), and Neal’s (2011) frameworks as well as from other writing and literacy scholars such as Selber (2004) and illustrates what rhetorical assessment of digital texts that gives attention to both the how and the why of composing might look like. The model emphasizes both composition process and product, it involves students in assessing their work at multiple points, it introduces and adapts terminology to describe composing in new media environments, and it uses its form as one way to communicate its meaning.

Figure 1. New Media Assessment Model.

The inner circle of the model represents composition product, assessed by instructor or students through looking at three criteria: first, fulfillment of purpose and direction to audience; second, the use of a multifaceted logic through consideration of layers of media; and third, the use of rhetorical and technical features for effect. I will discuss my rationale for choosing these three criteria below in further detail. The product itself is also assessed along with elements of the composition process, represented by the smaller circles on the outer ring. I use and adapt some of Allison’s (2009) labels for elements of the composition process, including participate, produce, and publish, and I add revise (instead of perfecting) and reflect. I have also designed my representation of the composition process in a circle with bi-directional arrows to represent the recursive nature of process, illustrating how students are free to move between stages in the composition process as they consider different aspects of the product in various and re-occurring orders.

As noted, my model also requires that students participate in reflecting on and assessing their work by setting goals related to their developing functional and rhetorical literacies (drawn from Selber’s 2004 tri-part framework for multiliteracies). In earlier versions of the model, I asked students to set what I called “process goals” and “product goals”; however, this terminology proved vague or confusing for some and resulted in overly generic goals. Using Selber’s terms functional and rhetorical instead of process and product focuses students specifically on the tools and technologies they use to compose (many of which students have not used before, such as video-editing software) and also on the rhetorical purposes for using these tools. I build development of critical literacies, the third aspect of Selber’s framework, into goal-setting and throughout the composition as a whole as students probe and question their own and their classmates’ processes and choices to use certain technologies and techniques over others. I use a set of questions (see Worksheet 1) to help students set initial goals, asking them to set functional goals, overarching rhetorical goals attending to purpose and audience, and specific rhetorical goals that address the parts and whole of the composition.

Worksheet 1. Setting Functional and Rhetorical Goals

Additionally, it is important to note that once students have set initial goals for developing their functional and rhetorical literacies at the beginning of a new media composition project, they must return to their goals at multiple points and revise them, which is one way of using the goal-setting process to develop critical literacies. Often, students are unsure of what a composition will look like at the beginning of the process and do not know which technologies and techniques will be best to use for their purposes. Goals set early on are useful in that they can direct students and give them concrete places to begin composing; however, goals often shift as a composition develops and changes. Thus I have students return to their goals weekly, revising and tweaking as they learn functional skills and refine the rhetorical purposes of their work. The model itself, with the reflection questions, can also be used as a tool to help students re-assess goals, to reflect on their composition’s effectiveness for a chosen purpose and audience, and to further develop critical literacies.

In addition to setting and revising goals for functional and rhetorical literacies, the model also asks students to think carefully about the product they create using the three criteria at the center of the inner circle in Figure 1 . The first criterion keeps students focused on the rhetorical principles of purpose and audience, which both Bearne (2009) and Neal (2011) included at the center of their frameworks. Second, authors of new media products employ a multifaceted logic by integrating layers of media such as visuals, sounds, music, animations, written text, and more. This criterion is drawn from much of the new media theory cited above, which argues for the use of new and evolving logics such as juxtaposition and association, for example. A multifaceted logic that considers layers could manifest itself through the juxtaposition and complex interaction of multiple modes of expression or it could be revealed through the choice to use only one or two modes of expression. Even using a single mode of expression (such as the visual) requires the use of what I am calling a multifaceted logic, as students contemplate which particular images to take up, think about how to employ them most effectively, and consider length, visual quality, color, and a multitude of other technical concerns.

The concepts of a multifaceted logic and considering layers of media are further explained through the text box in the lower left of the model in Figure 1 , which provides a short list of rhetorical and figurative devices that can be used to illustrate a multifaceted logic and could take form through different layers of media such as the written word, sound, music, images, animations, etc. This list includes the use of metaphor, metonymy, and synecdoche drawn from theories of visual rhetoric (Horn, 2004; Sorapure, 2006), juxtaposition, collage, and appropriation drawn from theories of new media logics (Janangelo, 1998; Sirc, 2010; Welch, 1999), sound and musical rhetoric (Halbritter, 2006), rhetorical appeals to logos, pathos, and ethos, drawn from Aristotle but also from contemporary new media work (Halbritter, 2006; Nichols, 2010), completion and reinforcement (Horn, 2004), and the inclusion of counterarguments drawn from theories of Rogerian argument (Young, Becker, & Pike, 1970). This list is by no means comprehensive, but provides students with tangible, specific ways they can begin to think and talk about employing a complex, multifaceted logic and incorporating multilayered effects into their work. Each item on the list, in fact, can be further explained and explored through class lessons and readings as students move through a course. The list draws student attention to ways of thinking and representing ideas that may be new to them. For example, one former student revised a draft of his web site on basketball by adding a multimodal metaphor to a tab, juxtaposing pictures of a player preparing to shoot a basketball, shooting, and following through with paragraphs of written text that documented the player’s moves through his career. These kinds of rhetorical decisions illustrate the power of a multifaceted logic and a growing consciousness of the relationships among layers of media.

Third, the model represents assessing a new media product through looking at the rhetorical and technical features used. Rhetorical features such as achieving purpose, demonstrating audience awareness, and using persuasive appeals can be realized through the use of technical features. Drawing in part from Bearne (2009), I provide a list of technical features for students to consider in the text box in the lower right of the model in Figure 1. These include but are not limited to principles of design such as contrast, repetition, alignment, and proximity (Williams, 2008), typographic and font choices, use of color, length, use of animations and movement, and citation and attribution. This list provides students with a starting place when they consider which technical features they can use in a composition and for what effects. In addition, each of these technical features could be addressed in more detail through in-class instruction and activities.

The final element of the model is a series of reflective prompts that guide students in enacting the kind of repeated, evidenced-based, rhetorical reflection that Neal (2011) recommended:

Please use the following prompts to guide reflection at multiple points throughout the composition process.

1. What goals did you set for this project? Why did you set these goals? Explain your rationale. Have the goals evolved and/or shifted, and if so, how and why?
2. Explain your chosen purpose and audience for the composition. Why did you choose this purpose and audience?
3. Explain how you have employed a multifaceted logic and considered layers of media in your composition.
4. Which rhetorical and technical features have you chosen to use in your composition, and why? What is the intended effect of these features?

The prompts ask students to reflect, at multiple points throughout the composition process, on their goals, their chosen purpose and audience, their use of multifaceted logics and consideration of layers of media, and their use of rhetorical and technical features. The prompts also ask students to articulate a rationale for the choices they make when they compose in any media.

ASSESSMENT IN ACTION:
REFLECTION ON THE IMPLEMENTATION OF THE MODEL

I have used a version of this model to assess new media video compositions in three 200-hundred level professional writing courses in 2011 and 2012 at the University of Michigan. Each class has helped me consider ways to revise aspects of the model, such as the terminologies I chose to guide goal-setting, and I have also had the opportunity to interview students about their experiences with video composition in our course and with the assessment model in particular. Below, I include excerpts of these interviews, focusing first on former student Ryan Schechtman and the self and instructor assessments of his video composition. I look first to Ryan because his reflections illuminate the interwoven development of all three of Selber’s literacies—functional, critical, and rhetorical. Yet Ryan’s experience is only one example of how the assessment model I describe can be taken up and applied in the writing classroom; thus I also offer several additional videos that illustrate how various students thought about and used goal-setting and the assessment model as they composed.

In response to the video composition assignment in English 229: Professional Writing, Ryan made a video advertisement for a pet day care facility. The course, subtitled “Rhetoric, Design, Technology: Composing 21st-century Professional Texts,” asked Ryan and his peers to compose and reflect upon a wide range of texts including a resume and cover letter, a collaborative business proposal, a web site, and a video. Below in Video 1, please view Ryan’s video “Karnik Pet Lodge” and listen at the same time as Ryan describes his video and talks about his goals for the project.

Video 1: Ryan talks about “Karnik Pet Lodge” and his goals for composition

The goals Ryan articulates in his reflective audio commentary were developed throughout his composition process, morphing as he learned functional video-editing skills and refined his rhetorical purposes. Here is the final written version of his goals for developing functional and rhetorical literacies, handed in with the final draft of “Karnik Pet Lodge”:

Functional Literacy Goals

1. Learn to use Windows Moviemaker and expand on my nonexistent base of video-making experience.
2. Use strategic zooming to control where the viewer’s focus is.
3. Smoothly transition between video and pictures multiple times.
4. Learn to edit video.

Rhetorical Literacy Goals

1. Create a video advertisement for Karnik’s website (that they will actually use).
2. Use a juxtaposition of images and text to explain the benefits of bringing your dog to Karnik.
3. Relatedly, I want to convey the message that it will be fun for their dog to the website’s viewers through pictures of happy dogs and videos of pups playing. Furthermore, I hope to correct some of the misconceptions pet owners often have about the quality of living in a kennel (or rather, show that Karnik is not a run of the mill, old kennel).
4. I aim to convey the message of caring, qualified, friendly, loving staff to viewers of Karnik’s website through appeals to logos, pathos, and ethos (especially pathos for the dog).
5. While I do strive to put forth a polished and professional image, I am more consciously focused on a tone of compassion, affection and playfulness. That is the type of person the dogs want; it should also be the type of person owners want taking care of their pet.

Although some of Ryan’s goals are fairly general and focus on the use of tools (for example, “Learn to use Windows Moviemaker” or “Learn to edit video”), others push him toward specific and complex rhetorical effects and authorial choices (“Use strategic zooming to control where the viewer’s focus is” or “Use a juxtaposition of images and text”). Ryan offered his own evaluation of his work toward his goals in a blog post on his writing process blog, reflecting on the choices he made. For example, addressing his “Use strategic zooming…” goal, Ryan stated,

I planned on having a project based largely on pictures to keep the viewers more focused on the text. However, when I put together a first draft totally composed of stills I felt that it needed something more active to keep the viewer captivated. At first I employed somewhat random zooms/pans, but it was suggested to me that I be more purposeful with my zooms and pans, so I was.

Through this comment, Ryan illustrates the development not just of functional literacies—of learning how to use zooming and panning with still images in the editing software—but of critical and rhetorical literacies as well as he questions and revises his use of techniques and technologies with audience and purpose in mind.

Using the assessment model and Ryan’s goals and reflections, I offered my evaluation of Ryan’s video to him with these written comments:

Ryan,

I enjoyed watching the final version of your Karnik video. Functionally, you have met all your goals: the zooming is strategic and helps focus what I’m looking at, and the transitions are smooth. Obviously, you have learned to use the software, too.

Rhetorically, I think this version does a fine job of juxtaposing images, text, and music. The written words you have added to this draft serve to organize the video and make the pictures more meaningful. I also like the combination of still and moving pictures and videos you use. You have also achieved your rhetorical goals relating to fun and caring through various images and the video’s organization.

My critiques are small: the cats get the short end of the stick with only one small moment. I kept waiting for them to return! And your chosen CC license isn’t on the video itself, which might be something to consider doing, especially if the video will be used. Also, do you want a short credits page to credit yourself and the folks in the video?

Great work! Overall, your video and your composition process exceed the criteria.

Through these comments, I point to the areas in which I see that Ryan has met both his goals for developing his functional and rhetorical literacies: using juxtaposition of images and written text, for example. I also refer to a few aspects of the assessment model, such as citation and attribution, which could be improved or expanded upon. The assessment model and goal-setting framework also give both Ryan and me the shared language with which we can discuss his work. Overall, Ryan exceeds the criteria laid out on the assessment model by moving through a complex composition process; setting, refining, and reflecting upon goals; and creating a video product that achieves his purposes.

Another lesson learned throughout the process of using this assessment model to evaluate student video compositions is that much of the student learning may not be immediately evident in the video products themselves; the same is true for Ryan. In his video interview, Ryan gives more insight into what he takes away from this project. Below, listen as Ryan discusses how he learned to use the terms functional and rhetorical literacies and how reflection impacted his learning.

Video 2: Ryan talks about using the terms functional literacies and rhetorical literacies

Video 3: Ryan talks about reflection

Of course, Ryan represents only one student’s experiences with this assessment model. In the videos that follow, I provide a brief glimpse into several other student experiences with video composing, goal-setting, and using the assessment model to think critically about process and product. Students such as Gabby found that goal-setting helped in guiding and directing their composition process in useful ways. But Diana expressed how she set goals for her video project “just to get by” and that these goals didn’t help or hinder her writing process, which suggests that some students may need further scaffolding to set goals that help them create high-quality products and are instrumental in their development as writers. Cameron explained how setting goals at the beginning of a composition process can be difficult because students often do not know where they are headed. This observation reinforces the need for students to return to their goals at multiple points in the composition process, revising and shifting them as they compose. Claire articulated how she appreciated the guidance provided in the model but also valued being able to design and guide her assessment in ways that were tailored specifically for her and her project. And as several students suggest, when the concepts and practices in the model are part of the fabric of the writing course as a whole and supported and scaffolded by lessons and in-class discussion, the model is able to function as a guide and a tool for student reflection and revision. Below, please listen as Cameron, Gabby, Diana, Ryan, and Claire describe their video projects, how goal setting did or didn’t play into their learning, how the assessment model helped them think about composition process, and how they used the assessment model while composing.

Video 4: How would you describe your video?

Video 5: How did setting goals contribute to your learning?

Video 6: How did the model help you think about your composition process?

Video 7: How did you use the assessment model?

As these students illustrate, the assessment model contains much that works well. It provides metalanguage for new media composition process and product—language that enables students to critically compose, revise, and reflect. Students can learn and apply terminologies such as functional and rhetorical literacies, musical rhetoric, or multifaceted logic, for example, to their new media compositions, and they can then carry concepts, practices, and awareness of their own composing with them to future writing contexts. The model also emphasizes student responsibility for composing and learning through goal-setting and reflection. Being able to set goals and self-assess is a practice that students may continue to find useful as they experience diverse writing situations in other courses or in the work world where audiences may or may not offer detailed, formative feedback on writing.

The model also has aspects that I continue to grapple with and question—aspects that need revision. At first glance, for example, some students were overwhelmed with the amount of information presented in the model, unsure of how to use it or where to begin. Other students mistakenly wanted to treat the model like a checklist, struggling to think about including everything listed in the boxes in their video compositions. To address these issues, might the model be broken down into smaller pieces and presented to students in stages as they compose? Others reported finding no use for some terms and concepts listed (like metonymy or reinforcement, for example) and composed in what appeared to be unconscious ways, only turning to the model and its concepts when directed to do so. Should students then be required to compose with techniques from the model, perhaps in smaller or lower-stakes assignments? Might I teach more mini-lessons on complex concepts, or might we look at and discuss additional models that use these effects? Still other students revealed less evidence of developing critical literacies as they made and evaluated choices about techniques and technologies—the result of my implicit instruction in this area was not always the conscious “informed critique” that Selber (2004) called for. So I continue to consider ways to help students focus more specifically on the critical: might students also set goals for developing critical literacies, even though the term critical may prove confusing? Might reflective prompts ask students to more directly question and critique the tools and technologies they use or choose not to use? I hope that questions such as these that remain may assist others as they think about how to adapt the model for their own purposes and contexts.

And as I myself consider the strengths and weaknesses of this assessment model and the ways I might revise it for future use, I realize that what began in an apprehensive moment as a search for new media assessment practices grounded in composition theory and responsive to students and the purposes and contexts of their writing has now become a representation of my pedagogy for new media composition. Embedded in the model is a rigorous composition process where students learn by doing and through collaborating. The model reflects and reinforces the need for instruction in metalanguage for new media that students can use to describe, develop, and become more aware of their own processes and products. And the model requires students and instructors to continuously reflect, providing evidence-based rationales for their choices. I have also become more aware that this assessment model and pedagogy is not static: Each time I use it, students and I find that it must morph and change to fit what we’re doing or may want to try, and we adapt the model to our local and contextual needs. I offer it, then, as a tool that I hope will push conversations about new media composition and its assessment forward, as well as link new media assessment to larger conversations in composition studies, both in scholarship and in teaching. I hope that readers will discuss the model with one another and with students, critique it, point out what is useful in it and what isn’t, change it, build on it, remediate it, and transform it for their local contexts. This kind of critical analysis and discussion between ourselves and our students is one important step on the way to reframing writing assessment for new media and for today’s composition.

REFERENCES