UNDULA

A dual-site experimental interface Designed to explore the communicative potential of physical movement

This project began with a fascination for how designing technology shapes our relationships, as changes in media inevitably transform how we connect with others. Living far from my family, I relied on technology to stay connected but always felt something was missing. Although technological advancements made communication more efficient, achieving a deep sense of genuine connection remained out of reach. The turning point came with the loss of my grandfather; unable to share my grief through words or texts, my sister and I found solace in silent, synchronous crying. This profound, non-verbal interaction sparked my interest in finding, framing, and asking critical questions about communication and how our definitions of it shape the technologies we design.

background

The early development of computer-mediated communication paralleled the rise of cognitivism, which viewed the mind as a computational mechanism, reducing the body to an input-output interface. This led to a conception of human communication as mere information transmission, akin to a machine sending signals from a sender to a receiver (1). This linear model, known as the information transmission metaphor, significantly oversimplifies human interaction by ignoring the dynamic, co-regulated nature of human communication. Recent studies in cognitive neuroscience and embodiment theory dispute this view (2), advocating for seeing communication as an emergent process that emphasises coordination and mutual adaptation through embodied interactions.

Recognising communication as a co-created dynamic rather than a static exchange of predefined messages has informed my practice-based PhD research project, which sought to discover new ways to experience connectedness online, both without and beyond words. Building on this understanding, I focused on designing an interactive system that underscores the fundamental role of synchrony in facilitating connectedness.

Our world is inherently wired for synchrony (3). Synchrony in animal and human behaviour refers to the coordinated and simultaneous actions, movements, or responses exhibited by individuals within a group or between interacting partners (4). This fascinating phenomenon occurs in many species through self-organizing, spontaneous, self-regulating processes (3). For instance, fireflies synchronize their flashing by adjusting their rhythm in response to others (5), while groups of male humpback whales can sing in unison for hours, collectively evolving their songs (6).

Dynamic coordination also occurs on smaller scales, such as in the synchronised courtship dances of seahorses, crucial for reproduction and pair-bonding (7). Humans, like other species, exhibit synchronous behaviour in both dyadic and group settings, demonstrating our innate pre-verbal ability to connect and communicate. In face-to-face interactions, we unconsciously coordinate various behaviours, including postural sway, walking in lockstep, speech patterns, eye gaze, facial expressions, and movements (8).
Synchrony is fundamental to human communication, beginning in infancy. Before developing verbal skills, babies engage in non-verbal communication with caregivers through synchronised movements, which are crucial for emotional bonds and healthy development. These early interactions create strong connections between babies and caregivers, laying the foundation for social, emotional, and cognitive development and paving the way for more complex communication, such as speech and language (9). Synchrony manifests in diverse ways and is acknowledged for its ability to strengthen interpersonal connections and rapport, acting as social glue (10).

In Human-Computer Interaction (HCI), movement synchrony is an emerging research area used to enhance collaboration (11), rapport (12), and human-robot interactions (13). Earlier studies focused on guiding people to synchronize with machine-generated rhythms through instructions (14), music(15), or gamification(16). For instance, participants tapped a key in time with a machine-generated rhythm (17). However, this approach is limiting as it neglects co-creation and self-regulation, making people synchronise with the machine rather than each other. My research argues that synchrony in social interactions, a dynamic and co-regulative process, is foundational to human communication (8). It establishes human sociality (18) and enables a state of temporal 'oneness' and connectedness (10).

Lindblom, J. (2020), 'A Radical Reassessment of the Body in Social Cognition' Frontiers in Psychology, Vol. 11 No. 987, pp. 1-16.
Gallese, V. (2003), 'The manifold nature of interpersonal relations: The quest for a common mechanism', Philosophical Transactions of the Royal Society B:Biological Sciences, Vol. 358 No. 1431, pp. 517-528.
Strogatz, S. H. (2003). Sync: The Emerging Science of Spontaneous Order. New York: Hyperion.
Schmidt, R. C., Morr, S., Fitzpatrick, P., & Richardson, M. J. (2012). Measuring the dynamics of interactional synchrony. Journal of Nonverbal Behavior, 36(4), 263-279.
Buck, J., & Buck, E. (1966). Mechanism of rhythmic synchronous flashing of fireflies: Synchronization of intrinsic membrane potentials of different-order pacemaker cells. Journal of Experimental Biology, 44(1), 1-14.
Payne, R. S., & McVay, S. (1971). Songs of humpback whales. Science, 173(3997), 585-597.
Vincent, A. C., & Sadler, L. M. (1995). Faithful pair bonds in wild seahorses, Hippocampus whitei. Animal behaviour, 50(6), 1557-1569.
Marsh, K. L., Richardson, M. J., & Schmidt, R. C. (2009). Social connection through joint action and interpersonal coordination. Topics in Cognitive Science, 1(2), 320-339.
Stern, D. N. (1985). The interpersonal world of the infant: A view from psychoanalysis and developmental psychology. London: Routledge.
Hove, M. J., & Risen, J. L. (2009). It's all in the timing: Interpersonal synchrony increases affiliation. Social Cognition, 27(6), 949-960.
Oertel, C., & Brandt, T. (2018). The impact of synchrony in collaborative task-solving. International Journal of Human-Computer Studies, 119, 13-23.
Kim, J. H., & Sundar, S. S. (2012). Examining collaborative persuasion in human-computer interaction: An observational study of a health campaign with varying levels of user control and interactivity. Journal of Computer-Mediated Communication, 17(3), 338-353.
Huisman, G., & van der Meijden, M. (2018). Robots that synchronize their movements with ours: enhancing human-robot collaboration by behavioral mimicry. International Journal of Social Robotics, 10(2), 285-295.
Kwon, D. S., & Kuzuoka, H. (2013). Gaze and movement synchronization among group members in face-to-face collaborative tasks. In Proceedings of the 2013 conference on Computer supported cooperative work, ACM, 1327-1336.
Chen, C. J., & Hung, Y. H. (2015). Analysis of the Effect of Human-Computer Interaction on Running Performance. In Proceedings of the 2015 International Conference on Advanced Computer Science and Information Systems, ACM, 71-74.
Robinson, R. B., Reid, E., Fey, J.C., Depping, A.E., Isbister, K., and Mandryk, R.L. (2020). Designing and Evaluating 'In the Same Boat', A Game of Embodied Synchronization for Enhancing Social Play. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). ACM, 1–14.
Rinott, M., & Tractinsky, N. (2022). Designing for interpersonal motor synchronization. Human–Computer Interaction, 37(1), 69-116.
Stuart, S. A.J. and Thibault, P. J. (2015). Enkinaesthetic polyphony: the underpinning for first-order languaging. In Lüdtke, U. M. (ed.) Emotion in Language: Theory – Research – Application. Series: Consciousness and emotion (10). John Benjamins, 113-134.

Developing undula

During my research I developed Undula, an experimental interface featuring two identical custom-made rocking chairs connected via distance. Each rocking chair is equipped with an accelerometer that allows to capture and sonify the chair’s rocking movement. By rocking on the chairs participants interact with and adapt to one another through a responsive soundscape created and shared by them over distance.

The soundscape was created in collaboration with sound designer Frederick Robinson. The soundscape imitates the sound of ocean waves, capturing the undulating motion of nearby and distant wave movements. The soundscape's spatial distribution creates an immersive experience, with sound shifting between front and rear speakers based on chair movement.

The prototype development followed agile principles with iterative prototyping, where the Undula interface underwent multiple testing phases in controlled environments and public exhibitions, with continuous feedback shaping each new iteration.

Prototype 1: Wooden Chair with Tension System

Prototype 2: Center of Gravity Experiment with Seat on Semicircular Base

Sketching and 3D Modeling for Final Chair Prototype Development

Final Chair Prototype Development: CNC Cutting Process

Final Chair Prototype Development: Construction in Progress

Final Chair Prototype in Action: Testing Dual Chair Interaction

Final Chair Prototype in Action

Project Outcomes

The Undula interface successfully enabled research participants to coordinate movement without visual contact or explicit instructions, fostering voluntary and autonomous interaction. Some participants reported a heightened sense of connectedness, showcasing this technology's potential to foster deep, meaningful bonds in remote settings.

This project challenges the traditional emphasis on explicit information transmission, advocating for a more inclusive approach to digital interaction. By focusing on co-creation and the dynamic, embodied nature of human interactions, it redefines communication technology. This research not only advances Human-Computer Interaction but also provides valuable insights for designing technologies that are more human-centered and emotionally resonant.

_{The Data Imaginary: Fears and Fantasies. K.Moline, A.Goddard, B.Davis, (curators). Griffith University Art Museum, Brisbane, Australia, 1 July - 18 September 2021. Photo: Carl Warner}

🔗

Thesis Access Link:

Client:
The University of New South Wales, Australia

Supervisors:
Prof. Mari Velonaki
Assoc. Prof. Katherine Moline
Prof. Katsumi Watanabe

Grants:
(1) University International Postgraduate Award;
(2) ADA HDR Essential Costs of Research Funding;
(3) HDR Completion Scholarship