ABSTRACT

As part of the “Digital Is” Initiative funded by the MacArthur Foundation, the National Writing Project organized a group of 11 teachers and researchers in the summer of 2010 to explore how multimodal forms of writing could be assessed. We began this work by posing a key question: What would the assessment of digital writing look like if we began with conversations between writers and readers, students and teachers, children and adults? In this chapter, we report on the ways in which we’ve begun addressing that question, specifically by presenting five domains—context, artifact, substance, process management and technique, and habits of mind—that link the language of assessing multimodal writing with acts that drive the creation and reception of digital texts.

While the assessment of digital and multimodal writing is a challenge for large-scale, standardized assessments, this chapter comes at the question of assessing digital writing from a different angle. We begin by asking: What would it look like if the language of assessment was closely aligned with the language used by the creators and readers of digital compositions? What would the assessment of digital writing look like if we began with conversations between writers and readers, students and teachers, children and adults?

These questions emerged from the community of writers and teachers working with the National Writing Project’s Digital Is... Initiative. Funded by the MacArthur Foundation through its Digital Media and Learning Initiative, the National Writing Project’s Digital Is... Initiative sought to explore the evolving nature of practice in the teaching of writing as teachers, composition instructors, mentors, and writers themselves come to understand the evolving nature of digital writing. Through Digital Is, the National Writing Project (NWP) organized a group of teachers and researchers to take up the challenge of exploring how multimodal forms of writing could be assessed when the focus of assessment was to inform and improve learning. Called the Multimodal Assessment Project (MAP), this group worked together over 18 months to surface the kinds of conversations that teachers and students felt contributed to their development as writers and to incorporate feedback from numerous other teachers and students. Through this process, the MAP committee refined a set of five domains that links the language of assessing multimodal writing with acts that drive the creation and reception of digital texts.

We are the MAP committee, and in this chapter we argue that considering the domains of (1) artifact, (2) context, (3) substance, (4) process management and technique, and (5) habits of mind as vital for the assessment of digital writing offers the possibility that the language of assessment can inform—and build upon—discussions more often associated with interaction, instruction, and text creation than with evaluation. Taken together as a full set, these domains name interrelated areas of interest and learning significant in students’ growth as digital writers.

Because these domains were developed as part of the NWP’s Digital Is… Initiative, they have a resonance with the guidelines and outcomes developed by the Conference on College Composition and Communication (2009), National Council of Teachers of English (2009), and Council of Writing Program Administrators (2011). Yet the MAP domains also begin at that intimate level of student–teacher and writer–reader interaction. This chapter defines and situates our five domains in relationship to the statements from these leading national organizations and the research literature on writing assessment. The chapter also offers examples from across K–16 and out-of-school contexts of how each domain can be used to help writers and teachers talk about the value of a particular digital work.

THE MAP DOMAINS

The MAP domains sketch out spaces where teachers and students, writers and readers, designers and game players interact when creating multimodal compositions. The domains overlap in some aspects, and no one domain provides a complete account of the writing or composing processes used when creating digital or multimodal artifacts. But considering (1) artifact; (2) context; (3) substance; (4) process management and technique; and (5) habits of mind does offer multimodal authors and those providing feedback and evaluating multimodal works with a framework for discussing both a single piece and a writer’s development over time. Although we provide examples that illustrate how these domains work with real multimodal pieces of writing later in the chapter, we sketch the general contours of each domain immediately below:

• The artifact is the finished product. Audiences expect artifacts to convey a coherent message with a clear focus created through an appropriate use of structure, medium, and technique. Artifacts incorporate elements from multiple modes, and are often digital, but do not have to be—they may be analog works (e.g., texts that incorporate both writing and drawing). They identify the connections among resources, composers, and ideas and may demonstrate habits of mind such as innovation, creativity, and critical stance.
• Context is the world around the artifact, around the creation of the artifact, and how the artifact enters, circulates, and fits into the world. Authors attend to the context of a multimodal artifact when they make design decisions related to genre or to an artifact’s intended uses. Given their purposes, authors consider the affordances, constraints, and opportunities, given purpose, audience, composing environment, and delivery mode.
• As a domain, substance refers to the content and overall quality and significance of the ideas presented. The substance of a piece is related to an artifact’s message in relationship to the contextual elements of purpose, genre, and audiences. Considering the substance of a piece encourages authors to think about elements such as quality of ideas, quality of performance, credibility, accuracy, and significance.
• Process management and technique refer to the skills, capacities, and processes involved in planning, creating, and circulating multimodal artifacts. Creating multimodal products involves the technical skills of production using the chosen tools, but it also includes larger project management skills as well as the ability to collaborate with others in diverse and often interactive situations. Over time, individuals learn to more effectively control the skills and manage the processes of producing and circulating digital content.
• Habits of mind are patterns of behavior or attitudes that reach beyond the artifact being created at the moment. They develop over time and can be nurtured through self-sponsored learning as well as teacher-facilitated activities throughout the process. Examples include creativity, persistence, risk-taking, mindfulness, and engagement. Habits of mind can also include an openness to participatory and interactive forms of engagement with audiences.

The development of these MAP domains is, in part, a response to the ways in which writing assessment has been far too often associated with timed, decontextualized exams, and where inter-rater reliability has trumped concerns about context and validity. Kathleen Blake Yancey (1999), Brian Huot (2002), George Hillocks (2002), Norbert Elliot (2005), and Sandra Murphy and Yancey (2008) have each traced aspects of the historical, ideological, and practical consequences of the dominance of inter-rater reliability in 20th century American writing assessment. The development of the domains could also be considered a response to formulaic approaches to the teaching of writing (e.g., the five-paragraph theme) that have developed along with these standardized forms of assessment. Patricia Lynne (2004) and Asao Inoue (2005) have argued that these assessments as well as the curricula and pedagogy that operate with them do not acknowledge the diversity of forms of writing in the real world and do not attend to context, audience, and purpose. Drawing on the work of teachers and students, Carl Whithaus (2005) and Michael Neal (2011) show that digital and multimodal forms of writing push back against these decontextualized approaches to evaluation and assessment.

Of course, there are a great variety of forms in non-digital, non-multimodal writing that also extend beyond the boundaries of traditional writing assessment. Although scholars have taken significantly different theoretical approaches, work in genre studies over the past few decades has consistently demonstrated that language (and thus writing) varies widely in relation to audience, purpose, situation, and setting (e.g., Bawarshi, 2003; Bazerman, 2002; Christie, 1987; Cope & Kalantizis, 1993; Miller, 1984; Swales, 1990). Digital and multimodal writing further emphasizes the variety of forms and the demands for situational elements to be considered during evaluation. Digital writers can choose among a plethora of platforms, tools, communities, and forms to accomplish their purposes, with new options being created all the time. The growth and variety of options for digital and multimodal writing put a high premium on authors’ attention to situational elements. Yet situational elements are not easily accommodated in standardized assessment processes. Coupled with the ways in which digital texts can produce near instant interactions with readers, the vast variety of forms means that digital writing assessment poses a complex set of problems.

As school-based, professional, and leisure forms of writing become more multimodal and more digital, the terms for evaluating writing and writerly skills need to shift. As this collection shows, richer, more situated approaches to assessing multimodal writing are being developed, particularly in post-secondary environments. Models that make writing assessment more contextual, such as those advocated for by Pamela Moss (1994), Bob Broad (2003), and Broad et al. (2009) enable thoughtful, nuanced assessments; they sketch more-developed portraits of student abilities than those created when traditional, timed, standardized writing assessments are used. This chapter reports on one national effort to develop a language of evaluation that works not only—or even mainly—in the realm of large-scale writing assessment or in post-secondary writing classrooms, but rather in the realm of K–16 classrooms, after-school programs, and even out-of-school contexts. Below, we outline the five MAP domains in more detail and provide illustrative examples of how each domain can be used to facilitate discussions with writers. We conclude by reporting on how the process of linking the language of assessment with the language of text creation and interaction is a “game changer,” because instead of abstract outside standards shaping the value of a text, situated discussions of a work become the major determining factors when assessing that piece’s value.

ARTIFACT: CHEESE, THE MOVIE

Produced in an after-school video club for fourth and fifth grade students, Cheese is about two heroes, Jack and George, who break into a high security fortress to recover (steal?) a mysterious object known as the “cheese.” The 3 1/2-minute action movie is lighthearted and fun. Using a script written the previous year by a returning student, the crew shot the movie quickly in the first weeks of the school year. Much of the time spent to complete the film was in “post-production.” Early versions of the video were shown to the other video club participants throughout the editing process and feedback was generally quite positive. Still, the editor made lots of changes, particularly in tightening up the cuts between shots and the addition of sound effects to complement the visuals. After countless viewings, the editor finally declared it done—an artifact.

Figure 1. Cheese the movie: Opening Screen. Full film available at: http://digitalis.nwp.org/resource/3404

Within the MAP Domains, the “artifact” is the final consumable (readable/viewable) product that stands on its own, can travel across space and time, and offers readers a coherent message through an appropriate use of structure, medium, and technique. The artifact can take on many multimodal forms and comes in a variety of mediums, both digital and analog. There is a chain-like relationship between the artifact’s message, structure, medium, and technique, where each link is equally important to the strength of the completed artifact, yet, the links build off each other. In this section, the student video production Cheese will illustrate how the domain of artifact can be used to help understand, assess, and communicate the strengths and weakness of a piece.

One of the central ideas of the artifact domain is the message or the content of the composition. As we’ll see in Cheese, the validity or power of a message may be judged differently by audiences depending on their expertise with a subject (“action films”) or perspectives on the events represented. The message in Cheese is fairly clear, yet incomplete; it is an action movie that lacks a premise. In the opening shot, Jack casually eats an apple and says to George, “Let’s go. If we leave now, we’ll be there when it closes.” Next, we see two guards in sunglasses holding guns. One asks the other to borrow his lighter. We see that it is for a cigar which she “lights.” Ambushed by Jack and George, the two guards are knocked out and they take their guns. In a comic twist, they also take their sunglasses and George even takes the cigar from her mouth before entering the building. They secretly slip into a garbage pushcart and the janitor unknowingly wheels Jack and George past seven more guards. The two heroes then jump out of the cart and start shooting, win the firefight, and race up to a door that says “PRIVATE! DO NOT ENTER!” They drop a hand grenade and run. While fleeing they see the object of the movie-- the “cheese”-- sitting on a shelf. They grab it, an alarm goes off, and they sprint out of the building. Once outside, the whole world shakes to the explosion. Safe and sound with the loot in hand, Jack and George celebrate their victory.

The video leaves the viewer with many, many questions: What is the “cheese?” Who are Jack and George? Are they stealing the cheese or recovering it? What is the building? If it “closes,” then is it a store? bank? We see guards, but who is the villain that has the cheese? Did the grenade cause the explosion or did the alarm trigger it? From the artifact itself, viewers don’t know the answer to any of these questions and without any additional support, explanations, or background from the authors, the message of the movie is unknown. Cheese dispenses with these details that might push the plot along and give structure to the film and instead delivers action—like a James Bond film without the meetings with “M.” Cheese feels like the ending to a movie more than a complete movie. For this reason, the overt message of the film is clearly in need of work.

However, the covert message is quite clear—the movie is all for fun and entertainment. And it is fun. The running, fighting, bombing, and shooting should seem violent but instead it all looks comical, not just because the guns are clearly play toys and smiles are hard to hide, but because the entire situation is impossible: knocking out the guards with a shove, hiding in the garbage with nothing covering them, winning a close-range shootout while outnumbered seven to two.

Cheese taps into the structure of traditional action movies and this is one of the strengths of the artifact. There are lots of action movie motifs, including guns, sunglasses, smoking, grenades, and explosions. The heroes have to get through layers of obstacles—strong-arming the outside guards, sneaking past the inside guards, winning a shootout, and using brute force to get through the private door. Just like other action movies, the compound of the villain is destroyed in one final explosion and, of course, the heroes win. To understand how structure and meaning fit together, it is beneficial to look at how structural problems in Cheese affect the meaning at the end of the movie. The way that the artifact presents the story, Jack and George stumble across the cheese while taking cover before the grenade detonates. This doesn’t make sense; wasn’t the grenade placed to get into the private room? Conventional action film structures would suggest that the grenade was used to enter the room where the coveted cheese is located, but instead the grenade doesn’t go off until after they’ve taken the cheese and even after they’re outside. As a result, it appears that the explosion was the result of triggering the alarm (which was caused by taking the cheese from its position.) The artifact’s structure and message work together and structural problems change the meaning in the movie.

Another important consideration when assessing the artifact is to see if the authors exploited the advantages of and avoided the limitations of the medium, as all mediums have different affordances and constraints when constructing representations (Eisner, 2002). The affordances of video are that it is multimodal, employing combinations of linguistic, visual, aural, gestural, and spatial modes. An action video, in particular, should utilize the visual and aural modes the most and Cheese does just that. The linguistic mode (print, speech) is used sparingly in Cheese, yet effectively—such as the sign on the door (“PRIVATE!”) and the subdued small talk from the guards before they are surprise attacked; however, once the action begins, the visuals, gestures, and sounds are the primary modes of representation. Cheese tells the story mainly through visuals and, as proof, is equally easy to follow with the sound completely off. Although the authors did not choose the medium (they made a video because they were in an after-school video club), they did create a story that took advantage of the medium to create a fast-moving narrative that is engaging during the entire length of the movie.

To build an artifact, authors must employ technical competencies that are often specific to the tools and media. Techniques may include using content in innovative ways, building upon other’s work to generate new ideas, using specialized composing tools, and more. There is a clear relationship between the medium being used and the techniques that accompany it. Video is a complex medium for children to work with because of the long nature of the video-making process, the interaction of the multiple authors, and the technical tools used to craft the story. Students in the after-school program produce videos in a complex, three-stage process (pre-production, production, and post-production) similar to the one used in major motion pictures. The quality of the final artifact depends on the mastery of a great number of technical skills including scriptwriting, prop construction, acting, directing, camera work, video and sound editing, and more. Two particularly significant tools used are video cameras and video-editing software and both require a steep learning curve to master. Video making involves multiple authors—scriptwriters, a director, a cameraperson, actors, editors, and more—and the final composition is arguably more collaborative than any other text students write in classrooms.

As to be expected of such a complicated task, there are plenty of technical issues in the video Cheese. Shortcomings include low-quality props (a dry erase marker used as a cigar), dialogue too soft to be heard well, laughing during “fight” scenes, and loud noises as the actors “secretly” climb into the garbage cart. In the shot with the outside guards, the viewers can clearly see the cameraperson and production crew in the reflection of the window. In other shots, we can see non-actors in the corners of the background. Attempting to capture “everything,” the novice cameraperson sometimes tracks any character that moves instead of framing the shots artfully and letting the characters move within the frame.

Video often requires a discerning eye to notice technical proficiency, and, technically, Cheese does many things well: The editing is generally tight and matches the quick tempo of the content, camera angles vary nicely from shot to shot, with a few exceptions there is good continuity between shots, the acting is spirited, and the sound effects are effective in pushing the story along. In addition, Cheese has several technically complex shots that require extensive planning, communication between participants, and synchronized execution. For example, the director choreographed a sequence that integrated in-camera effects (shaking the camera to simulate the impact of an explosion) with actors pretended to ride out the earthquake-like rumble. Combined with sound effects in post-production, the shot works incredibly well but the complexity is often lost on the viewer.

In many ways, Cheese pays homage to the larger genre of the action movie, but the domain of artifact also includes internal practices of citation and attribution. Within an artifact, authors may identify or cite contributors and resources. These acknowledgements or attributions may use an identifiable system that can range from formal bibliography (i.e., MLA, APA, Chicago Manuel of Style, IEEE style sheets) to the informal embedding of external links to minimal or no attributions. Video and film have their own systems of attribution in the form of “credits,” which Cheese does include. In fact, over a third of the total screen time of Cheese is credits. The movie begins with a quick title credit and concludes with a significantly longer ending credit sequence that acknowledges all members of the production team, naming the director first, then the two “stars,” the “extras,” and the production crew. Although the artifact does use credits in a fairly systematic way, no credit was given for the images or the sound effects gathered from the Internet. The students clearly value the contributions of people, but they are still learning to deal respectfully with sources and intellectual property.

Learning to compose is often about learning to look at the artifact alone as a portable document that attempts to convey a message without any additional support, explanations, or background from the authors. We can’t generally accompany our pieces, explaining our intents or correcting others’ readings. Examining the artifact closely—its message, structure, medium, technique, and attribution—is an important step in understanding, assessing, and improving the piece. The contradictions we find are important opportunities for developing meaning and clarity, whether through conferencing during the revision process or looking at a completed piece as a way of improving future pieces.

Although large-scale decontextualized assessments raise concerns for many in the profession (Hillocks, 2002; Huot, 2002; Lynne, 2004), the practice of assessing, revising, and critiquing the artifact has an important place in learning how to compose effective multimodal pieces. As an assessment domain, artifact encourages an examination of a piece’s message, how that message is structured, the use of a medium’s affordances to convey that message, the techniques the creators employ, and methods of attributing parts of the piece drawn from other sources. But as an assessment domain rather than a complete schema for assessing a work, artifact exists in dialogue with the other four domains: context, substance, process management and technique, and habits of mind.

CONTEXT: TWEETS AND POEMS ABOUT
THE THINGS THEY CARRIED

When thinking about evaluating multimodal writing, context is a domain that helps explain how the artifact fits into the world. Context encourages us to ask about the environments surrounding the creation of the artifact and how the artifact enters into that world. For instance, if you take the series of microblog posts in Figure 2 out of context, at best, they appear to be confusing and at worse they could be considered nonsensical. However, if we begin to fill in the context behind this twitter stream, the short pieces of writing take on a different significance—both as individual pieces and as a larger collaborative project.

Figure 2. Microblog Posts from a 9th Grade American Studies Class.

The tweets in Figure 2 were created in a ninth grade American Studies classroom, where students used Twitter to microblog in character as they listened to selected chapters of The Things They Carried read aloud. They were assigned to “platoon groups” of five “soldiers” in which each student’s task was to imagine the thoughts of his or her assigned character (in a kind of real-time role-playing), tweet those thoughts, and respond to the tweets of other “soldiers” in his or her assigned “platoon.” The Twitter feed and selected lines from the book were later used by students to create first-person poems of war. The hope was that the activity would allow students to read deeply and experience some level of empathy with the characters.

In most classroom activities, the artifact usually takes precedence as the primary object of assessment, sometimes followed by the process used to create the artifact. In the “Microblogging in Character” activity, both were important, but context—how deeply students entered into the constraints, affordances, and opportunities of the environment and tools surrounding the creation of their artifacts—was perhaps even more important with relation to student learning and the empathy they developed with characters.

The final artifacts, the poems, are not multimodal. They are simple print-based constructs, and yet the creation of the poems required student interaction with and construction of multimodal texts, and the assignment also required multimodal ways of understanding. Students had as “givens” the purpose, audience, composing environment, and delivery mode of the activity. Their purpose was to express how a soldier might feel in the midst of combat; their immediate audience was their peers and teachers (an additional later audience would be veterans they would interview and with whom they would share their poetry); the composing environment was the classroom and a small collaborative group of peers; and the delivery mode was the assigned poem. One group of students composed a piece called “A Poem of the Vietnam War”:

A Poem of the Vietnam War

I survived, but it’s not a happy ending.
Carrying diaries like lost souls on our backs.
Humping pictures of ones we may never see again.
Hot weather, dense thicket,
Blinding rays of sun
Make it seem almost impossible to escape.
Driving thirst
Overwhelming all of us.
Praying and begging for the kiss of rain to be our only joy.

I survived, but it’s not a happy ending.

We all have our own problems at hand.
Setting those aside is the only way to get through this hostile land.
Yet, some raging,
Others stressing,
This war is just outrageously crazy.
The thought of how they expect me to be,
The way to react,
Worries me deeply.

I survived, but it’s not a happy ending.

In thinking about context as an assessment domain, we circle around the tweets that students used as a form of brainstorming and as a way of opening up their thoughts and feelings about characters in The Things They Carried. We also circle around the poems that they wrote. Both the tweet stream and the poems are artifacts, but in thinking about the context of assessment we move toward new questions: How were these inter-related artifacts created? What moved students to write both the tweets and the poems? As an assessment domain, context enables a teacher to talk about how students worked in collaborative groups. She can represent student learning to students, parents, and administrators as the process that evolves from a microblogging activity through small-group discussions and writing into the final collaborative poems. Context, then, becomes a space for dialogue, a space where we want to see the assignment, the brainstorming activities, and the “final” artifacts as a developmental chain.

The poems speak, but the learning—and the record of the learning—goes back to the assignment, to the microblogging activity, and to the reading itself. Because we are examining this twitter stream and some of the student poems in relationship with the domain of context, it is worth seeing the assignment used to move students from reading The Things They Carried to tweeting about the short story to writing their collaborative poems. The original assignment appears below:

Twitter Poem Activity

Overview: In this activity, you will draw from The Things They Carried, from the class Twitter feed, and from other sources as necessary to write a collaborative poem about war.

1. You need to use one of the following quotes as a repeating hook:
   • "What you have to do is trust your own story."
   • "I want you to feel what I felt."
   • "The thing about remembering is that you don't forget."
   • "Stories are for joining the past to the future."
   • "What sticks to memory often are those odd little fragments."
   • "I survived, but it's not a happy ending."
   • "What would you do?"
   • "A true war story is never moral."
2. Your poem needs to incorporate tweets, either coming directly from your twitter feed or coming as a thread of an idea from your twitter feed.
3. Your poem needs to use a beautiful or powerful combination of words to create images in the minds of your readers. A poem uses compacted language to create mental pictures. In a poem, every word counts.
4. Your poem should reflect a sense of empathy towards the things soldiers went through during wartime and toward the soldiers themselves.
5. Your poem needs a consistent theme, with a beginning and an end, and the theme should be authentic to the time period and the story.
6. Your poem doesn't have to rhyme.
7. It needs to have a great title that fits your poem, that isn't a label or description, that attracts the reader's attention, and that is grounded in the text of the poem.
8. This poem needs to reflect the work of every member of your group. This is not to be a one-person assignment.

In addition to considering the assignment, as an assessment domain, context encourages developing an account for how the students responded to the constraints and affordances of Twitter. With its 140-character limit, Twitter presented students with the challenge of being concise and with making their language compact. Those who met that challenge had strong and beautiful language for poetry. The real-time synchronicity of hearing the story and simultaneously tweeting as though present in the action was for students constraining, requiring their full attention and immediate response, but when students did meet this challenge, their language was fresh and poignant. In their creation of poems, students made use of the story to pull from it their responses to the horrors and drudgery of war; they made use of their collectively created Twitter feed, and they used each others’ ideas to compose poetry and write group poems. Students entered more deeply into empathy with their characters as they entered into the constraints and affordances of the activity’s context.

How does a teacher assess student immersion into context and their understanding of it? A teacher could look at the success of the final product or artifact; this could provide some evidence. Perhaps more helpful might be student reflections and discussions of the constraints and affordances of the tools and environment, a process that aligns with most scholarship in multimodal assessment (Alexander, 2007.) In a Ning discussion forum, students were asked to discuss useful and not so useful tweets, and why they thought as they did. They were also asked to identify and discuss tweets that did a good job of embodying the thoughts of a character and showed empathy, and say why they thought these were good tweets. In this way, multimodal assessment criteria development became collaborative and more student-centered. Overall, these sorts of questions have the potential of tapping into student knowledge and awareness of the context of the activity and deepening that understanding for those who have preliminary or partial understanding.

The importance of building understanding and empathy in The Things They Carried activities highlights how classroom learning through writing reaches beyond “writing as a skill” to “writing to learn” about other concepts. The integrity of concepts—whether humanistic concepts such as empathy or scientific concepts such as how semi-permeable membranes facilitate diffusion—is often divorced from writing assessment. That is, the substance of an argument or the accuracy of historical facts or scientific data is often not highlighted in the scoring rubrics for large-scale writing exams.

SUBSTANCE: ABRAHAM LINCOLN’S BIOGRAPHY TOLD VIA GOOGLE EARTH

By insisting that substance is a domain that must be considered in writing assessment, the MAP group is breaking with the long-standing tradition of separating form from content, of assuming that the quality of writing can be assessed without considering its substance. Although substance is embedded within an artifact and relates to the artifact’s context, the substance of a multimodal piece of writing has its own resonance, its own importance. As a key domain for both assessment and learning—particularly in academic settings—the term substance refers to the content, overall quality, and significance of the ideas presented. The substance of a piece is related to an artifact’s message in relationship to the contextual elements of purpose and audiences. Considering the substance of a piece encourages us to think about four main areas: quality of ideas, credibility, accuracy, and significance.

In multiple grade levels and content areas, students are regularly asked to create reports, biographies, or essays on famous individuals. These assignments might appear as part of a history or English class, and are a classic example of educational expository writing. Although the merits of this type of assignment can be debated, its existence in education is undisputed. Teachers may require students to complete these projects because “it’s something that has always been done” or because biographies and expository report writing are genres explicitly mentioned in content standards.

In the case of one California K–8 school, teams of English and history teachers determined that students struggled comprehending expository historical text and observed that these same students were challenged to produce effective expository writing including summaries and reports. Like many middle-school students, these 8th graders had been introduced to this type of writing multiple times since 4th grade, but they still needed additional support.

As part of the 8th grade United States history curriculum, the team of teachers at this site designed a Famous Americans unit. As a culminating activity toward the end of the school year students would construct a research report on a famous person from American history. The students had ultimate freedom to choose their person of inquiry and could conduct their research using online resources, library books, and textbook materials previously taught in the classroom. The history teachers would assist students in finding historically accurate information, while the English teachers would teach the research report writing genre. Both departments would provide individualized support to students on comprehending expository text and historical information through shared and guided reading activities.

This particular school site had also recently received an Enhancing Education Through Technology (EETT) grant and as part of that grant students were paired up with teachers through an inverse-mentorship model. Students and teachers co-designed technology-infused lessons in partnership, where students who had been trained in a technology elective class provided technology support, and teachers supplied instructional knowledge. Once designed, the student would be in the teacher’s classroom to teach the technology components of the lesson to fellow students and provide technical support as needed.

In light of the technology grant, the English and history teachers decided to have the students present their research reports through a multimodal artifact. Students would still write a traditional text-based research report, but would also construct a multimodal demonstration of the same information. Students could choose any software for the multimodal artifact, but the teachers assumed students would likely use presentation software, such as Microsoft PowerPoint or Apple Keynote. As students completed their research reports, the technology elective teacher suggested to the students that they try using Google Earth as a presentation tool. This particular software would allow students to display text, images, video, and links to external resources in a medium that also provided a geo-spatial construct. For example, students could describe the childhood life of their famous American through a placemark containing text, images, and video positioned with an aerial view of the person’s hometown displayed in the background. As a result of this suggestion, nearly half of the 8th grade class created a Google Earth tour for their multimodal presentation. Figure 3 is an example of the types of projects produced by the students.

Figure 3. Google Earth Abraham Lincoln Presentation.

The student who created the multimodal artifact above chose Abraham Lincoln as his person of inquiry and faced a daunting task in converting the content of a standard research report to a Google Earth Tour. Through five placemarks and a title folder, the author takes the reader through a geospatial and multimodal tour of Lincoln’s life. During this process, the author needed to consider many of the contextual elements described in the previous section: audiences would encounter their information in a different setting and different platform with different expectations for presentation; audiences would navigate the information in whatever order they wished and would expect it to hold together in each of the paths; audiences would expect the artifact to take advantage of the platform affordances.

The five placemarks also allowed the author to take his original research report and remix the content so that it was displayed in a logical manner. For example, the portions of his research report that dealt with Lincoln’s early years are found in placemarks over his birthplace and first adult residence in Springfield, Illinois. Similarly, information regarding Lincoln’s presidency and assassination are found in placemarks located near the White House and Ford’s Theater. Each of these placemarks displays images, text, and links to web sites that were not found in the original research report. Some of the text, such as “the Google Earth screen shot is an aerial view of...” is critical to the reader accurately understanding the Google Earth version of the story. In other placemarks, the images, text, or links provide additional information not easily presented in a text-based research report. For example, in the assassination placemark, the author embedded a YouTube video from the National Park Service that provides a tour of Ford’s Theater. The author also eliminated structures commonly found in a research report, such as introduction and conclusion paragraphs, because these would not have been a logical part of a Google Earth tour. Combined together, these additions and deletions assist in transforming a research report into a coherent multimodal artifact meant for a different context.

These issues provide great fodder for discussions with students about the role of context. But in addition to context, the author had to consider the four areas of substance—quality of ideas, credibility, accuracy, and significance. The assignment, to present a research report, demands attention to substance in addition to technique, process, or skill with platform and its tools. And so with a focus on substance we can read many of the choices the student made differently.

While creating the Google Earth tour of Abraham Lincoln’s life, the author demonstrated credibility in his artifact through the thoughtful choices of different elements. These included images directly complementing the information in the text and locating placemarks so that they were displayed with appropriate and visually interesting three-dimensional buildings in the background. This choice of images and placemark location helps to convey the author’s use of critical thinking, intelligence, and diligence in creating the artifact. For example, to be able to use the effects above, this student needed to use HTML mark-up, which was not normally taught in the technology elective course. Students, however, took the basic information their teacher provided and—through online research, trial-and-error, and persistence—created multi-modal artifacts that effectively embedded and displayed credible information.

As part of this research project, students were required to use multiple sources. Throughout the artifact, it is evident that the author was very knowledgeable about Abraham Lincoln’s life. The text flows and makes logical sense when compared with commonly known facts about Lincoln. The artifact, however, also demonstrates areas where the student could improve accuracy and significance by using additional sources and ensuring his writing does not mirror the sources he has used too closely. For example, on the title folder the author only cites online sources. His textbook, primary-source documents used in history class, and additional non-digital sources are not cited. Many of the online sources directly reference Wikipedia, bringing into question the range of his search and the validity of his citations and accuracy of the information. This multimodal artifact demonstrates a need for additional instruction or coaching regarding citations and the effective use of many and diverse sources.

In many ways, the Google Earth tour demonstrates significance as a multimodal artifact because it pushes the envelope in terms of what could be done to present a research report. When designing the project, the teachers had assumed students would create a slideshow presentation or perhaps even a digital movie. They did not consider that the students would produce Google Earth projects. However, this particular tool provided an element that neither the slideshow-creation application nor the video-production application could. Google Earth allowed the author to tell the story of Abraham Lincoln’s life through text, images, and video on the geographic backdrop of Lincoln’s life.

PROCESS MANAGEMENT AND TECHNIQUE:
THE DIGITAL MIRROR COMPUTER CAMP WEBSITE

Learning about process management and technique was central to the experience of a team of adolescent girls as they created a web site while attending the Digital Mirror Computer Camp, a 4-day residential experience for middle-school girls collaboratively developed and delivered by women faculty and graduate students in the Rhetoric and Writing doctoral program at Bowling Green State University. Now in its fifth year, the camp helps girls enhance their attitudes about and aptitude for technology in both their social and academic lives through direct instruction in functional, critical, and rhetorical digital literacies (Selber, 2004). With this goal in mind, camp facilitators provide training in web authoring, digital imaging, and video and audio editing. Each year, new campers develop a web portfolio that includes links to reflective multimodal artifacts including blogs and digital videos about their experiences with technology, and the camp concludes with a showcase session for families to celebrate the girls’ work.

The camp is a digital outreach experience designed to create a safe space for young women to experiment with technology in a collaborative, supportive environment, and many of the participants return for a second and even a third year. Given the desire to expand the digital skill sets of the campers and move them from individual work in their first year to collaborative work in their second year, returning campers have frequently collaborated on a multimodal project in ways that align both with the domain of process management and technique and with specific rhetorical contexts and purposes.

In this example, a group of nine second-year campers worked together to design a camp web site aimed at future participants and their parents. Because process management and technique emphasizes planning, creating, and revising multimodal artifacts, the collaborative development of the Digital Mirror web site (see Figure 4) is an appropriate model to document how those new to digital composing processes must learn to coordinate tasks, share knowledge, and take collective responsibility for achieving their rhetorical goals.

Figure 4. Homepage for Digital Mirror Camp Website.

Grouped in teams of three, the campers pooled their labor and their skills to create a professional identity not only for the camp but also for themselves as audience-aware multimodal composers. As participants in the camp familiar with camp activities that would appeal to both campers and their parents, the group began to plan both content and design. Relevant design choices included the color scheme for the site (including shades of pink, orange, and blue), along with the development of a camp logo, which includes an image of a cellphone, an important digital tool for this age group.

The development of a multimodal artifact like a web site demands a different type of composing process, which includes the management of digital assets such as HTML files, images formatted for web viewing, and, in this case, the creation of multiple digital video and audio files the campers recorded, edited, imported, and finally exported for web delivery. For all of these technical elements to work together to make the site functional, the girls had to develop a range of digital skills, including knowledge of file formats, directory structure, and server management, along with shared responsibilities in using tools such as Adobe Dreamweaver and Photoshop as well as Apple iMovie and GarageBand. As with any group of novice designers, some technical aspects are not as strong as others and detract from the professional quality of the site, such as wobbly video footage, or even the instance of typographical errors in some pages. Nevertheless, the campers began to master the skills and techniques involved in digital production in a collaborative environment that encouraged a mentoring model among the campers themselves.

Although these functional skills are crucial, as Sonya Borton and Brian Huot (2007) suggested, “all instruction in the use of digital and nondigital composing tools, and all assessment of multimodal compositions, should be tailored to teaching students how to use rhetorical principles appropriately and effectively” (p. 99). The Digital Mirror web site achieves this goal through design strategies that include the alignment of text and image in addition to links to the online work of past campers (blogs, web sites, and videos). These technical features enhance the purpose of the site: to promote the camp to both an adolescent and a parental audience. This dual audience calls for a range of rhetorical strategies, and thus the three groups focused specifically on developing the various portions of the site that appealed to multiple age groups, relying on the interface design, textual content, image editing, and video production to achieve their goals. For example, the parental portion of the site includes content about the qualifications of the camp facilitators as well as detailed information about the types of digital work the campers do and the fact that the adolescent female participants are chaperoned throughout the experience. In addition, the video segment on that portion of the site enacts scenarios in which the girls use personal entry devices to access the upper floors of the residence halls where they are housed. The camper-oriented portion of the site, on the other hand, relies on rhetorical appeals, such as student testimonials, that some of the individual members of the nine-member team wrote themselves to stress the entertainment value of the camp (even as the girls learn to compose in new media spaces). Similarly, the embedded video segment features teamwork activities, the camp fieldtrip experience, and the overall camaraderie among the girls and between the girls and the camp facilitators.

That the girls ultimately saw the site as a collaborative effort is evident through their first names appearing at the bottom of the homepage as collaborative designers, and the fact that the entire site was created within a 3-day period suggests the level of teamwork and time management necessary to develop a finished product to share with parents and camp facilitators. In this way, the site represents the process management and technique domain because girls occupied different roles, sharing responsibility within and across the three groups as authors, editors, videographers, and even actors within the video clips on each portion of the site.

To better enable that collaborative process, the team of nine initially engaged in team-building activities such as the game Peanut Butter River, in which the team uses a limited number of squares to collectively cross an imaginary river. Despite these efforts in preparing the girls to function as a team there were also some less successful aspects of the project, including the reluctance and overall timidity of the girls to publicly share the site with the parents and families at the camp’s final showcase session. This serves as an important reminder that although adolescent multimodal composers may understand the need to make rhetorical choices in content, design, and modality, learning to communicate and rationalize those choices requires equal emphasis on the part of digital literacy educators as part of any formative and summative assessment process.

Just as the Digital Mirror Computer Camp is designed to foster functional, critical, and rhetorical digital literacies, it also fosters the teamwork skills required for individual and collective success as multimodal composers with a very specific purpose and a very compressed deadline (4 days), and provides opportunity for the self-assessment of the impact of digital composing on girls’ emerging online identities. Such an approach also counters the larger cultural assumptions about girls’ use of technology as restricted to their roles as passively uncritical social consumers as opposed to actively critical professional producers. Although the girls select somewhat predictable color schemes and patterns for the camp (pink and polkadots), they also include a series of links to sites about girls and technology, suggesting an overall awareness that their own efforts are aligned with larger social goals and the multiple audiences who share those goals. As the camp facilitators noted

these community components challenge the perception that technology is isolating and make the collaborative identity of the camp more appealing to prospective campers. Technology, therefore, becomes more than just entertainment, but something that allows these girls to make an impression on other girls. (Blair et al., 2010, p. 155)

As the Digital Mirror web site shows, the campers achieve this goal through managing the process of planning, developing, and distributing rhetorically effective digital content.

The domain of process management and technique encourages attention to how writers develop their abilities to manage complex processes and the technical skills needed to be effective composers in multimodal environments. This domain is more explicitly about the processes that writers are going through than is the domain of context. Process management and technique is also more reflective and more about skill or technique development than Context is; however, as a focal point, process management and technique is less reflective, and less big picture than habits of mind is.

The domain habits of mind draws heavily on the reflective; it relates to the insistence in the Council for Writing Program Administrator’s (2011) “Framework for Success in Postsecondary Education” that learning to write effectively in college and beyond is not only about skills acquisition but is also about developing approaches to learning that are both intellectual and practical.

HABITS OF MIND:
“LETTERS FROM THE INTERNMENT CAMPS”

The habits of mind domain encourages us to look at ways in which writer–designers engage in persistent, active learning. In this section, the Voice Thread of fourth grade students exploring issues of life in “Letters from the Internment Camps” will demonstrate how habits of mind may frame and help assess the products and processes involved with digital compositions. According to Educause (2009), Voice Thread is a commercial, educational technology that works as a media aggregator. The web site allows people to post multimedia artifacts for educational feedback. Responders can add to the voice thread by means of microphone, webcam, keyboard, or telephone. The application may support completely new types of sharing and presentation, and also support critical and creative responding to these new types of texts.

From the site description of “Letters from the Internment Camps” (see Figure 5), the fourth grade students explore an historical issue—the placement of Japanese-Americans into internment camps during WWII. The design of the learning project makes use of realistic imagining (if I were interned…) in the letters and drawings that become artifacts for exposition. The creative writing of historical letters reaches a high point in collaboration when actual internment-camp survivors respond and become part of the ongoing digital composition. The Voice Thread classroom manages to teach writing and history as a habit of the creative mind—and thus stimulate the truth of student knowing in and beyond the class.

Figure 5.Slide from “Letters from the Internment Camps” Voicethread.

The purpose of a learning framework for habits of mind is not necessarily to push the field closer to a psychological assessment so much as to offer theoretical conceptualizations within which assignments, courses, and programs may be imagined and implemented. Therefore, when qualities from the domain of habits of mind are applied to digital compositions and the “Letters from the Internment Camps” Voice Thread in particular, teachers and students are challenged to reconsider their practices, behaviors, and intentions during the mental processes called into play by the learning experience. For example, the children’s Voice Threads demonstrate how the situated experience of composing and reading letters aloud make students more aware of history as imagined participants. The imagination of the experience becomes more real as they adopt their personae to the internment camp letters, and goes further as the responses to the Voice Threads from teachers and internment camp survivors continue to make history alive and present in and out of the classroom.

Admittedly, teachers are not typically trained as psychologists, psychiatrists, or brain researchers, and may thus feel wary in applying habits of mind as a key domain for digital composition. Frameworks help teach educators and students about necessary conditions and resources, and the frameworks for habits of mind are intended to unpack and situate how we evaluate multimodal composition by focusing administrators, teachers, and students to recognize the evidence of thinking in composition, and to re-vision what we do to cultivate learning. “Letters from the Internment Camps” suggest that students achieve a sense of digital citizenship beyond the historical exploration their fictional letters encounter. Indeed, not only the participating students but also readers of their work gain a better understanding of how their ethical behavior deepens a sense of cultural belonging and societal questioning in a multimodal context. Using habits of mind in assessing compositions connects literacy growth more directly to analyzing behavior during the composing processes and changing existing activities to stimulate the best learnable situations. Almost all of the writers who participated in the Voice Thread show a focused consciousness of how their current activities fused with their current behaviors to create analyses of historical issues.

When digital composition is assessed in light of the habits of mind that produce and receive created artifacts, literacy education moves from valuing traditional critical thinking as an object and point of terminal thinking to encouraging more reflective emphases, because the assignments we offer and the processes we cultivate synthesize new combinations of meaning making. More and more technology improvements mirror this increased “mash up” or “read/write” approach as assessment begins to present questions about motivation, disposition, attitude, learning styles, thinking predilections, and personae. New emphases on habits of mind are helping the field move from collecting ideas as gradable objects toward conceptualizing learning as developing new mental processes more closely aligned with emerging technologies of literacy. It is not hard to hope that fourth grade student writers placing themselves in internment camps may prevent such a thing from happening again, or at least create an ethos that might halt such future segregation. Habits of mind, as a domain for assessment thinking, gives writers and readers stronger awareness of how their own behaviors, processes, and products reflect changing worlds of learning.

Teaching habits of mind as a key part of a writing process is necessary because composers need to assess decisions that enable them to hold true to ideas they are discovering and authenticating. Assessing process thinking means considering the effects generated from habits of mind and linked to composing decisions. Writers can raise consciousness about how their responses to challenges influence their growth when directing audience attention, when changing purposes in process, when adding or subtracting media, and when the blending of genre and modes is necessary in multimodal composition. Assessing habits of mind may create a metacognitive sense of authoring that actually reflects much of the writer’s identity.

Troy Hicks (2009) pointed to identity as a learning key enacted when we “ask students to create public digital writing personas at the same time they know they are doing work for a grade” (p. 107). Hicks continued by explaining that self-reflection—added to formative and summative assessment of digital composing—is “an act of identity formation, a twenty-first century skill that students need to have as they represent themselves across a variety of online communities” (p. 107). This notion of identity as it relates to habits of mind is powerful, according to Hicks, because technologies (like file management, tagging, software tools, and aggregators) enable students and teachers to track their digital work over time and across media. Finally, framing aspects of identity with habits of mind creates a method to foreground the student writer and the learning processes instead of letting the work or grade become the learning object.

CONCLUSIONS

At the beginning of our work together as a committee reflecting on multimodal assessment, we felt pinched by our experiences with high-stakes writing assessment. As a central pillar of school reform and institutional monitoring policies, it felt far away from our experience as teachers, learners, and digital writers. But by starting with the question, what do we talk about when we talk about how we can get better?, we surfaced issues that mattered to us as writers and media makers. What would it look like if the language of assessment was closely aligned with the language used by the creators and readers of digital compositions? It would mean that our assessment conversations would need to go far beyond what conventional rubrics and assessment programs aim to touch.

Eve Bearne (2009) noted that if multimodal projects are to survive the curricular squeeze so common in K-16 contexts, then assessment of multimodal products must “shift away from summative assessments of learning towards processes of continuing assessment or assessment for learning” (p. 19). For any type of multimodal assessment to aid in learning, it needs the flexibility to address both the context and the developmental capacities of the learner. By honoring the domain of the artifact, but also expanding our conversations to include context, substance, process management and technique, and habits of mind, we have several entry points to assessment, allowing us to focus on the domain best suited for the developmental focus of the learner. As noted in Because Digital Writing Matters (2010), the interest of technology in schools is not the technologies but “in looking at what the tools do for engaged learners/writers/creators” (p. 143).

Each example within this chapter could be discussed through the lens of several of the domains. We discussed the Google Earth project using the domain of substance; however, a focus on habits of mind could forefront the development of a young man who reached beyond the assignment to learn HTML mark-up, and certainly habits of mind can apply to the Twitter project. The nine young women at the Digital Mirror Camp demonstrated strong rhetorical, technical, and process skills in creating their web site, yet they still had much to learn about the context skills of promoting their work in the vast space of the web, where social circulation and curation processes influence whether a site is seen or unseen. The more we looked at examples of young people’s work, the more we listened to conversations among authors and teachers, the more we asked our colleagues what was important for specific young authors in helping them “get better,” the more the language of the full set of domains—and not just the narrow language of tools or single artifacts or demands of singular assignments—seemed vital.

In the first blossoming of the Internet, many educators saw multimodal products as more “authentic” expressions of youth culture, created through novel processes and shared by youth out-of-school through personal and social networking spaces and sites. We now see that multimodal composing is composing, and needs to be seen in the fullness of its uses and rhetorical possibilities. If multimodal composing in schools is to go beyond the occasional special project created for the teacher, then educators are compelled to guide and assess this skill development in language that both classroom teachers and their students can more easily comprehend and apply. The MAP Domains, as well as the projects described in the other chapters in this collection, document movements toward forms of multimodal writing assessment that are richly situated and responsive to learners as well as those who want to measure learning in context.

REFERENCES