Analysis and visualization of multi-source traces of human interaction

Analysis and visualization of multi-source traces of human interaction

What is a digital trace of human interaction? In the foundational and theoretical paper (in French, my translation) Traces, interaction traces, learning traces: definitions, computer models, structuring, processing and uses(1), with Alain Mille, we define it as follows:

“[A trace] is a temporally situated sequence of observed events, which is either an interaction between humans, mediated in various ways by computers, or a sequence of actions and reactions between a human and a computer. This trace is eventually re-playable, in which case, it becomes dynamic. It is digital since it is a recording of actions performed on a computer or a digitized version of a video (showing humans interacting or showing a screen capture during this interaction).” Lund & Mille, 2009.

Defining what we mean by a digital trace is necessary for the four other pie-chart themes in this section.

In regard to Models of learning and affordances of traces, I highlight three papers. First, the guidelines that are proposed in GESTALT: a framework for redesign of educational software(2) rest upon analyzing the digital tools involved, extrapolating from the tools to sequences of associated human actions, and examining the degree of coherence between tools and actions and the relation to pedagogical goals and the educational situation.

In 2009 I co-founded the company Cognik. It focused on personalization of multimedia use (e.g. educational games, Nickelodeon videos, MyMTV music). The second paper I highlight, Giving parents feedback on individual children’s progress during on-line educational games: (site and company are no longer active) was presented in Shanghai during the CSCL 2011 conference. China was one of the countries in which our educational games were marketed. Cognik’s feedback was based on an individual child’s trajectory but interestingly, parents in the audience wanted feedback that compared their child to other children in their area, and the world.

Thirdly, in Technological affordances for productive multivocality in analysis(4), we illustrate in what ways Tatiana (Trace Analysis Tool for Interaction Analysts) can assist the comparison of analyses of a shared corpus, carried out by researchers from different academic traditions. For example, it’s possible to transform a trace of a computer-mediated small group problem solving session that abstracts up from individual events to higher-level threads, where events and threads are conceptualized differently, depending on the research tradition.

I looked more closely at such trace transformations in the theme Connecting levels of analysis through the paper Gaining Insight by Transforming Between Temporal Representations of Human Interaction(5). Here, we transformed data on multiple interactional linguistics annotations of a short conversation between three different representations in order to connect micro level indicators of emotion and macro level emotive descriptions of sequences of interaction. In the paper Effects of Expertise, Mapping and Tutorial Format on User Experience when Gesturing with the Nintendo Wii(6), results showed that experts have better performance on making new gestures and become competent earlier than novices. In addition, strong direct mapping from physical gesture to virtual gesture during particular gameplay situations facilitates players’ understanding and manipulation of the WiiTM. We suggest how these results can aid designers of gestural interfaces in general.