Experiment 3 — Related Works: StoryBeads (2013)

Listen to this article

Background

As part of the Creation & Computation course, each student submitted 5 experiments in creative computing. To accompany our explorations, we were tasked with selecting one work or grouping of works (artwork, design project, research artifact, technology innovation, etc.) that was related to the theme or tools of the current experiment.

The theme for Experiment 3 was Tangible Interfaces, and we learned how to use Processing with Arduino, as well as different types of sensors.

The following post gives an overview of the work I selected, and how it relates to the theme. If you would like to see what I created for Experiment 3, check out my post on "Footwork".

Description of the work

StoryBeads are necklaces made of small computer “beads” capable of transmitting, displaying, or storing images. They are wearable computers used for constructing stories by allowing users to sequence and trade story pieces combining image and text. The beads communicate using infrared light, which allows for the trading of digital images from one bead to the other. The network consists of a chain of beads connected wirelessly that can communicate with their two nearest neighbours. Each necklace is essentially a database of images distributed across a network of beads. Inter-necklace communication allows a community of users to share stories digitally by exchanging physical beads between necklaces, or by beaming them from necklace to another necklace. As images move between users, new image descriptions are added to provide a historical context. (Barry, Davenport, and McGuire 2000)

The concept above was originally conceived by Barbara Barry, Glorianna Davenport and Dan McGuire in a conference paper for the IEEE International Workshop on Networked Appliances in 2000, and variations of this concept exist. I actually learned about this through another conference paper which discussed another variation of the StoryBeads from 2013 by Lizette Reitsma, Andrew Smith, and Elise van den Hoven.

This latest version of the StoryBeads differs from Barry, Davenport and McGuire's version because it utilizes audio instead of video messages, and they attempt to make the technology less intrusive than the predecessor. The 2013 StoryBeads addresses the need to preserve cultural knowledge for the next generations. It's focused on preserving Indigenous Knowledge (IK) of the South-African BaNtwane culture, focusing on their oral traditions and beadwork.

This newer model consists of two components:

1. eBeads: Physical handcrafted beads, and
2. StoryTeller: A recording/playback device

According to the conference paper, the device would work as such:

"eBeads function as Tangible User Interfaces (TUI’s) that activate the StoryTeller to either:

1. record and associate a story with the eBead
2. playback a story that has previously been associated with that particular eBead, or
3. delete an associated story when the eBead is dropped through a hole in the surface of the StoryTeller.

Once the eBead has passed through the hole the eBead would be ‘empty’ again. Individual eBeads served as narrative elements and multiple eBeads could be strung together to form a larger piece containing story extensions. 

The electronic circuitry consisted of a radio frequency identification (RFID) reader, an audio recorder/playback/storage circuit, and an Arduino controller. Custom-written software monitored data received from the RFID reader, comparing the detected eBead with a list of previously recorded stories.

If a match was found, the stored recording was played back over the built-in loudspeaker. If a match was not found, a prompting message was played over the loudspeaker and sounds picked up by the built-in microphone saved to non-volatile memory. The audio recording was terminated when the RFID reader no longer detected the presence of the eBead." (Reitsma, Smith, and van den Hoven 2013, p.81 - 82)

Why or how this relates to issues in physical computing that are being investigated in Experiment 3

The conference paper does a great job of outlining how the beadwork of the South-African BaNtwane is used as a tangible interface, which support the user in interacting with digital data by means of an interface in the physical world:

"The proposed use of traditional African artifacts as tangible programming elements explores cultural artifacts as Tangible User Interfaces (TUI's). Personal tangible objects are objects with which the user has an existing association. Natural mappings may therefore already exist between specific digital information and specific personal tangible objects. This concept may be extended to cultural tangible objects.

Cultural artifacts play an important role in Indigenous Knowledge systems, serving as symbolic frameworks that guide appropriate behavior amongst community members in various situations. Taking those objects as a foundation for TUI’s may help the user understand digital information that has now been associated with them."

Creators of the work, and a little about their backgrounds

Here is a quick look at the creators of the original StoryBeads and the latest version:

2000 version

• Barbara A. Barry, Ph.D., is a human-computer interaction (HCI) researcher who studies how interaction with artificial intelligence (AI) impacts human intelligence, communication and behavior. Dr. Barry has made fundamental contributions to AI, developed and evaluated intelligent interactive software agents, and collaborated on large-scale implementations of new technologies to improve health and education. Dr. Barry's work focuses on how technology enables (or disables) human agency in health care and how to create productive, ethical and evidence-based use of AI within a learning health care system. (Mayo clinic n.d.)
• Glorianna Davenport is Principal Research Scientist at the MIT Media Laboratory in Cambridge, Massachusetts where she founded the Interactive Cinema group (1987-2004) in 1987 and most recently the Media Fabrics group that she directs. She holds a B.A. in English from Mt. Holyoke College, South Hadley, Ma. and a M.A. from Hunter College in NYC. Trained as a sculptor and documentary filmmaker, Ms. Davenport has achieved international recognition for her work in the digital media forms. Davenport's research explores fundamental issues related to the collaborative co-construction of digital media experiences, where the task of narration is split among authors, consumers, and computer mediators. Davenport's recent work focuses on the creation of customizable, personalizable storyteller systems which dynamically serve and adapt to a widely dispersed society of audience. (Massachusetts Institute of Technology n.d.)
  
  
• Dan McGuire is the Director of Population Health at Providence Community Health Centers in Providence, Rhode Island. He holds a B.S. in Electrical Engineering from M.I.T., and an MBA from The Tuck School of Business at Dartmouth. He is currently focusing on improving population health, provider utilization, and patient satisfaction through analytics, technology, and process improvement. (McGuire n.d.)

2013 version:

• Lizette Reitsma is an Associate Senior Lecturer in Design for Sustainability and Social Change at Malmö University in Stockholm, Sweden. They are interested in how design can play a role in making the debate around sustainability more democratic and heterogeneous. Within this context, they are looking at the sensitivities that should have a position in such a design space, such as humbleness and respect towards other ways of knowing. (Reitsma n.d.; n.d.)
  
  
• Andrew Smith works at the Council for Scientific and Industrial Research Meraka Institute in Pretoria, South Africa. (Smith n.d.)
  
  
• Elise van den Hoven is Professor of Human-Computer Interaction at University of Technology Sydney and at Eindhoven University of Technology, the Netherlands. In addition, she has an honorary appoint­ment in Duncan of Jordanstone College of Art and Design, University of Dundee (UoD). Van den Hoven's research interests span different disciplines, including interaction design, human-computer interaction and cognitive psychology, which come together in the research program she leads called Materialising Memories. In Materialising Memories design research is used to study and support people in their everyday remembering practices. (van den Hoven n.d.)

Works Cited / Bibliography

Barry, Barbara, Glorianna Davenport, and Dan McGuire. 2000. “StoryBeads: A Wearable for Story Construction and Trade.” In Proceedings of the IEEE International Workshop on Networked Appliances 2000, 11. Newark, NJ, USA: MIT. https://mf.media.mit.edu/pubs/conference.html

Hoven, Elise van den. n.d. “Elise van Den Hoven.” Research Gate. Accessed November 17, 2020. https://www.researchgate.net/profile/Elise_Van_den_Hoven

Massachusetts Institute of Technology. n.d. “Glorianna Davenport.” Interactive Cinema. Accessed November 17, 2020. https://ic.media.mit.edu/people/gid/

Mayo clinic. n.d. “Barbara A. Barry, Ph.D.” Mayo Clinic. Accessed November 17, 2020. https://www.mayo.edu/research/faculty/barry-barbara-a-ph-d/bio-20429245

Mazalek, Alexandra. 2001. “Tangible Interfaces for Interactive Point-of-View Narratives.” Master’s Thesis, Cambridge, Massachusetts, USA: Massachusetts Institute of Technology. http://zyzx.haust.edu.cn/moocresource/data/050422/U/803/pdfs/mazalek.pdf

McGuire, Dan. n.d. “Dan McGuire.” LinkedIn. Accessed November 17, 2020. https://www.linkedin.com/in/danielcmcguire/

Reitsma, Lizette. n.d. “Lizette Reitsma.” LinkedIn. Accessed November 17, 2020a. https://www.linkedin.com/in/lizettereitsma/?originalSubdomain=se

———. n.d. “Lizette Reitsma.” Research Gate. Accessed November 17, 2020b. https://www.researchgate.net/profile/Lizette_Reitsma

Reitsma, Lizette, Andrew Smith, and Elise van den Hoven. 2013. “StoryBeads: Preserving Indigenous Knowledge through Tangible Interaction Design.” In 2013 International Conference on Culture and Computing, 79–85. Kyoto, Japan: IEEE. https://doi.org/10.1109/CultureComputing.2013.22

Smith, Andrew. n.d. “Andrew Smith.” Research Gate. Accessed November 17, 2020. https://www.researchgate.net/profile/Andrew_Smith7

Smith, Andrew, Lizette Reitsma, Elise van den Hoven, Paula Kotze, and Louis Coetzee. 2011. “Towards Preserving Indigenous Oral Stories Using Tangible Objects.” In 2011 Second International Conference on Culture and Computing, 86–91. Kyoto, Japan: IEEE. https://doi.org/10.1109/Culture-Computing.2011.24