Back in 2006, Bella Dicks, Bambo Soyinka, and Amanda Cody wrote an awesome essay, Multimodal Ethnography. The link here is to Sage, but you can get a paid-for copy through the Sage Premiere data base at the Kean library.
The focus is primarily on how the choice of media for data collection and representation create different versions of the community/field setting you are studying. This is because different media set up "readers" to make different meanings. These different meanings arise from the fact that each medium allows readers access to (different) sets of semiotic modes associated with that medium.
Modes = "abstract, non-material resources of meaning-making (obvious ones include writing, speech and images; less obvious ones include gestures, facial expression, texture, size and shape, even colour). . . Modes cannot be directly observed, for they are abstract resouces: they are rule-governed, codified sets of meaning-resources, involving the ida of formal 'grammars' - for example, the 'grammar' of film and that of writing . . ." 82
Media ="are the specific material forms in which modes are realized, including tools and materials.
So "writing" as a verb may be thought of as referring to the "modes" it entails, and "writing" as a noun would be the medium that invokes the modes associated with written texts. ** at least that is the way I am understanding this before going back to re-read Kress.
Before they get into the analysis of field-documentation for her project she makes a distinction between inherent features of the worlds we study (what is "there"); our fieldnotes/docuentations of what happens = our perceptions of what is "there" in the field. Data records are representations - already interpretations. They are not what is "there". What is "there" (if it is anything at all) is always becoming/disappearing - and our perception of it is always local, partial, filtered through our ideological positioning, and documented in light of our modes of perception + representation.
They also set up the idea that the field is always multimediated - and our records of what happened are always simplifications of that "what happened" = (and sometimes translations between/across modes).
Why is modality important for ethnographers to think about?
Dick et al make a compelling argument for the importance of attention to the modalities of data collection + representation (eg fieldnotes v photos v videos v video with sound etc) in their analysis of modalities invoked by different media (see p 88ff).
For example, they described writing as primarily employing verbal language (yeah, I know, this does an end run around the idea that it is an image - but let's let that go for now) and that the modality of photos is image. In comparing the primary modes of these media, we come up with something like the following.
Writing
allows other modes to be described in their absence ("colour, sound, shapes, textures as well as actions and movement")
appears as subjective/affords a particular perspective (that of the writer)
allows evaluation, emotional response
perspective can 'jump' =>allow for movement in time and place, and be multiple
Photograph
interpreted as objective =>what is really their- the "shaping" effected by the Machine (the camera) and the photographer is often overlooked
positioned within the perspective of the camera
fixed/static perspective
access to modes of physicality (colour, texture, shape, relationship)
89-90 They go on to compare the modalities available in writing/photography to what is made available in edited video (two shots with a split screen juxtaposed) and note that: far fewer modes are in evidence in the fieldnotes than in the video footage. Fieldnotes: narrative + syntax (which can describe other modes in their absence). But they also give us access to interpretive information that the video does not: they give "immediate clues to the writer's interests"/biases, expectations; afford clear sense of interpretation => so that interpretations that are implicit to the framing/justaposition/selection in the video - are (can be) explicit in the writing (and could be explicit in a video with voice over on a second run through?/more editing)
and so on about temporality + hierarchy (this part, 91) is pretty interesting = though audio recordings confound some of the discussion about speech v writing.
Sum up = "multimodal representation implies the need for careful consideration of the particular kinds of meaning afforded by the different media
Why is this important to our project?
The observations/reflections in this essay seem important to our project in a number of ways.
1. Because we are documenting online experiences/communities, all of our field data will be multimodal. It is important for us to remember that in every case ONLY part of our field will be mediated (there will always be someone on the other side of the screen from the images we capture - and part of their experience of the on-screen activity is what is going on is the space where their body is situated). This means that the experiences we are documenting will always have mediated and sensory components - which will be experienced through complex combinations of modes.
2. Part of our "field" will already be mediated. The on-screen data will be there as an object - available for observation as a "clone" of the object observed by our participants. The differences will be in terms of the machines on which they are viewed (phone, computer screen, ipad, reader . . .), the material-world,context of the reading, the perspective/context of the reader, and so on. These data will raise questions about whether screenshots and screen casts are in some ways closer simulations of what participants experienced than say => photographs, videos or write-ups of reading other texts? While at first glance this may seem a "seamless" representation of participant's experience = I see our ethnographies as taking apart that assumption.
3. Our representations will be partial in different ways than traditional texts- despite their multimodality or perhaps because of them? Because our representations will be hyper-linked and audio-visual, in some ways we can create simulations of what happens on-screen - but not the experience of on-screen (the life of the community) - and we must be aware of that.Contemplating the particular multimodal nature of our observations, documentation, and representations (interpretations) can allow us to account for some of the difference.
Whew. So that is pretty abstract. Just to nail this down, here is an action component.
1. As you collect your data (participate in your community), note YOUR context = physical environment, emotional responses, sensations, diversions and so on. A journal with dates/time that correspond to data would be good. Remember that your participation - as a member of your community - is one kind of evidence of how the actions on the bright side of the screen affect what shows up ON the screen.
2. Include inquiries about material-world contexts surrounding specific posts/on-line interactions. If multimodal/sensory ethnography is going to work for us - we need data to do the "connective" work.
3. Write up/describe the modes (abstract interpretive resources=writing, images, sounds, speech, spatial arrangements, colors, etc) + use of media within your field data. => Analyze the layout, images, use of sounds + video in the online spaces/material world spaces you are working with.
4. Start to write up some reflections (analytic memos) about how media/modes influenced your/your participants experience of your community.
I think this is just so cool. Let me know if you need a copy of the article.
No comments:
Post a Comment