Alexandra Mazalek & Paul Clifton
As we reflected on the commonalities and differences between the Lukasa and our own experiences with designing tangible narratives on digital tabletop, we realized we would not be able to convey the stories of the Luba peoples in our design, the way some tangible narratives “contain” prerecorded stories that are conveyed through audiovisual media. There appears to be little easily accessible Western documentation of the wealth of stories captured by Lukasa boards and the only textual description of a Lukasa “reading” we were able to find was from Memory: Luba Art and the Making of History by Mary Nooter Roberts and Allen F. Roberts. As a result, we began to design around the idea of the Lukasa as a performative mnemonic device. Based on a conversation with Lubangi Muniania, an art educator specializing in the visual and performing arts of Africa who, as a teenager, was intitated into the Luba art and culture, we realized that in order to effectively present this concept in a short amount of time, we needed to focus on a single component of the use of the Lukasa in Luba culture: genealogical records. Genealogy provides a universally accessible framework for storytelling and enables visitors to draw their own conclusions about the assignment of meaning to arbitrary elements of a form.
The piece uses a digital tabletop as large scale version of a Lukasa, on which visitors’ stories are created through a combination of tangible object and multi-touch interaction. As each visitor constructs their story, small animations representing their story’s components begin to play on an adjacent wall. The piece is intended to be experienced in small groups, where visitors complete the interactive session by sharing their stories with one another around the table, accompanied by the playback of the animated scene on the wall. As each group completes their session, their tabletop “Lukasa” is captured as a digital image and stored in an archive of every visitor’s creations. These conceptual maps, which represent the stories shared by visitors, are displayed in a slideshow on screens at the back of the exhibition space. As with the Lukasa, without the original creators to give meaning to these maps by narrating the stories they represent, the recorded images remain little more than abstract symbols to subsequent viewers.
Luba Memory Mapping
In developing the design for the interactive component of the Mapping Place exhibition, we investigated the way certain artifacts, particularly from the Luba peoples of Central Africa, capture narratives of the past, serving as conceptual maps that spatially encode elements of place, genealogy, political relations, and more. Representing both shared and personal histories, these artifacts can be seen as performative mnemonic devices: triggers for memory during the act of storytelling.
The Luba artifacts and practices that exist as conceptual maps — aids for memory and storytelling – are varied: from beaded necklaces and headdresses, to staffs, spears, scepters, and scarification.
We focused our design on a specific Luba artifact: the Lukasa, or memory board. The Lukasa is an hourglass-shaped, hand-sized wooden board that is studded with beads and shells and/or carved with ideograms.
These physical representations (beads, carvings, etc.) are symbolic, but also open to interpretation. That is, they do not represent the past in a static way; rather, each telling or performance with a Lukasa is unique. Roberts and Roberts call it a “generative reconstruction of the past”
and note: “although the Lukasa is learned, particular meanings are assigned on significant occasion in a specified locale for a given audience.”
Only the creator of a Lukasa, often a court historian, will know the originally intended meaning of its particular selection and spatial configuration of beads or carvings, and through subsequent readings.
As the board passes to new hands, it takes on new meanings to fit current circumstances.
Tangible Narratives
The Lukasa and other Luba mnemonic artifacts are especially relevant in light of emerging forms of digital media interaction known as tangible and embodied interfaces, physical interactive objects that couple input and output through the use of embedded sensors. The Synaesthetic Media Lab at Georgia Tech explores new forms of interactive storytelling and expression that we call tangible narratives.
Like the Luba artifacts, tangible narratives employ physical objects and surfaces to represent different story elements, including characters, places, and events. Both Luba artifacts and tangible narratives are not fixed: they employ the power of the abstraction of story elements and their relationships to create dynamic, evolving, and generative stories, which deliver custom experiences for each visitor or group of visitors.
Some tangible narratives are realized as sculptural artifacts or installations, while others make use of digital tabletops as a site for the narrative experience. In the latter case, aspects of the story are visually represented on the surface of a tabletop display or physically embodied with handheld interactive objects whose position and orientation can be detected on the tabletop display surface. For the Lukasa, the tangible and visual representations serve as a kind of conceptual story map. In the tangible narrative system, these representations also serve as controls for the digital story system. By manipulating these representations through multi-touch or tangible object interaction, visitors navigate the story space or trigger the progression of the story, for example through the audiovisual play-out of story components.
Tabletop tangible narrative systems and their associated media content represent only the computationally mediated layer of the storytelling experience. A second, even more open-ended layer of storytelling takes place around the tangible narrative system, as groups of visitors use the digitally mediated story elements to create further stories, shared orally the around the table in a kind of improvised performance.
Design and Development
The design of the piece was guided not only by the overarching goals for the exhibition, but also by audience and site related factors. Given the museum setting for the piece, we expected that visitors would typically interact for five minutes or less. Furthermore, we expected the main audience would be primary or middle school children on school field trips. As a result, we realized it would be necessary to convey the Lukasa-inspired story-mapping concept through a short interaction that would be both easy to understand and engaging.
Over the course of several design iterations, we developed an interaction sequence for the construction and sharing of stories that consists of four simple steps, as shown in Figure 3. As visitors approach the digital table, they see beads floating around the surface and a collection of physical objects shaped like shells on the edges. A visitor can pick up a shell and place it on the tabletop display to start building their digital story. This shell becomes the central node of their story, and seven icons appear in a circular “menu” around it, representing possible components of a story about family and place: man, woman, boy, girl, pet, home, and city. The visitors can arrange digital beads around the real world shell by dragging them onto the icons to assign meaning to them. For example, if a visitor wants to tell a story about their mother and aunt, they can drag two beads onto the “woman” icon one after another. They may choose to represent their mother with a red bead and their aunt with a yellow bead in order to distinguish them.
As each bead is assigned meaning, a small corresponding animation begins to play on a wall adjacent to the table. When more beads are added, the animations are layered to create a story scene. The visitor can arrange their scene as desired by moving the assigned beads around the central shell. The story is completed when the visitor removes the tangible shell from the table and the “menu” of icons disappears. Only the final arrangement of assigned beads remains and the visitor can verbally share their story with others around the table.
The visual and physical design of the piece draws on Luba art in general and the Lukasa in particular. Inspired by the cowrie shells in a Lukasa diagram shown in Memory: Luba Art and the Making of History,1 which represent royal and spirit capitals, we decided to center each visitor’s story around a single shell. The tangible shell serves as an interactive object that allows the visitor to create their story; when removed, a graphical representation of the shell remains underneath, surrounded by the beads that have been assigned meaning during the story construction. This cluster of shell and colored beads, , remains on the table as a graphic “map” of their story. Multiple visitors can create stories on the table at the same time, each with their own tangible shell, resulting in a larger story map of the group’s shared interactive experience.
Technical Implementation
The piece was developed on a ReacTIVision-based interactive
tabletop, which makes use of computer vision to detect both finger touches and tangible objects that have special visual markers attached to their undersides.2
The table uses the diffused surface illumination (DSI) technique to evenly light the surface with infrared (IR) light by attaching strips of IR LEDs to the edge of a sheet of acrylic impregnated with tiny reflective particles. When touched or when an object is placed on it, the infrared light is reflected towards a set of four modified PlaystationTM Eye cameras, which pass their images to a computer vision engine called Community Core Vision (CCV). CCV stitches the images together and interprets them to determine the location of fingers and objects on the surface. These locations are read by an application developed using the Unity3D game engine, which generates the images projected on to the surface and the walls based on the input from CCV and the current state of the application.
The form of the table and the tangible shell objects were designed in SolidWorks, a 3D solid modeling tool. The table was cut at the Advanced Wood Projects Lab at Georgia Tech from plywood using a CNC (computer numerical control) router, and the tangible shell objects were 3D printed on a Dimension SST 768 in the GVU Prototyping Lab, also at Georgia Tech. The objects were then finished by sanding and painting the plastic. The 3D printer reads a solid model file from a program like SolidWorks and builds the form layer by layer from an extruded filament of melted ABS plastic.
Having these tools as well as the students and faculty with the diverse skillsets required to use them, created an opportunity for students at Georgia Tech to construct installations like this one, learn to work with diverse teams, and to imagine, design and create new modes of interaction around systems that consist of many different software, hardware, and physical components.
Summary
The Lukasa-inspired interactive exhibit demonstrates one way in which emerging digital media interaction technologies can be used to revisit long-lived and deeply rooted concepts, such as the way Central African cultures map out their experience of the world in ways that are tangible, embodied, and performative. When experienced alongside the static and two-dimensional maps of Africa created by Westerners, we hope this interactive exhibit can expose Western audiences to a broader understanding of the concept, practices, and functions of mapping place and lived experience.
notes
1 Roberts, M. N. and Roberts, A. F., Eds. (1996). Memory: Luba Art and the Making
of History. New York, Museum for African Art.
2 Roberts and Roberts, 33.
3 Roberts and Roberts, 37.
4 Roberts and Roberts, 118.
5 Roberts and Roberts, 41.
6 Mazalek, A. (2011). Tangible narratives: Emerging interfaces for digital storytelling and machinima. The machinima reader. H. Lowood and M. Nitsche, Eds. Cambridge, MA, MIT Press: 91–110.
7 See reference for example:
Chenzira, A., Chen, Y. and Mazalek, A. (2008). Renati: Recontextualizing narratives for tangible interfaces. Proceedings of the 14th International Symposium on Electronic Art (ISEA ‘08), ISEA2008 Pte Ltd, 106–108.
8 See references for examples:
Mazalek, A., Davenport, G. and Ishii, H. (2002). Tangible viewpoints: A physical approach to multimedia stories. Proceedings of the tenth ACM international conference on Multimedia (MULTIMEDIA ‘02), ACM, New York, NY, USA, 153– 160; Mazalek, A. and Davenport, G. (2003). A tangible platform for documenting experiences and sharing multimedia stories. Proceedings of the 2003 ACM SIGMM workshop on Experiential telepresence (ETP ‘03), ACM, New York, NY, USA, 105–109;Mazalek, A., Winegarden, C., Al-Haddad, T., Robinson, S. J. and Wu, C.-S. (2009). Architales: Physical/digital co-design of an interactive story table. Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI ‘09), ACM, New York, NY, USA, 241–248; Tanenbaum, J. and Tanenbaum, K. (2011). The reading glove: A non-linear adaptive tangible narrative. Proceedings of the 4th international conference on Interactive Digital Storytelling (ICIDS’11), Springer-Verlag, Berlin, Heidelberg, 346–349.
9 Roberts and Roberts, 141.
10 Kaltenbrunner, M. and Bencina, R. (2007). Reactivision: A computer-vision framework for table-based tangible interaction. Proceedings of the 1st international conference on Tangible and Embedded Interaction (TEI ‘07), (Baton Rouge, Louisiana), ACM, New York, NY, 69–74.