Learning with Sound
I heard the loud snapping sound in my right hip. I had been trying to reach the drain-plug in the bathtub from a challenging distance. I had never heard such a sound before, but it was similar to the snapping sound an overstretched elastic cord makes when it ruptures. Instantly, I felt something inside my right hip. I froze in fear and focused my total attention to the area.
“Did you notice you only relaxed your muscles when I did deep breathing with you, but not when you were doing it on your own?” said my physiotherapist, who was examining my piriformis muscle in my right buttock, which turns out to be what I tore from overextension.
“Really? I had no idea. I thought I was releasing all of my muscles,” I said, wondering how I could have been mistaken. I was genuinely focusing on whole body relaxation and feeling some release of tension when I was slowly exhaling as instructed. I was feeling severe muscle pain, but I was quite impressed with her insight. The discrepancy between my feelings and her observation got me thinking about knowing the unknowable and possible crossmodal correspondence between vocal sounds, imagination, and motor responses.
An illustrative example of crossmodal correspondence, especially the sound-motion association, is music, which often induces more emotional response in listeners if there are some visual kinematic cues or bodily gestures by performers or a conductor. Visual kinematic cues have been found to influence felt emotion, the perception of phrasing and musical tension, the perception of emotional expression, the overall appreciation of the performance, and even the prediction of piano competition winners. Another example of sound-motion association can be found in the martial arts. When participants are doing forms or breaking blocks, they often shout a mono-syllable (e.g., Ya!, Ha!, etc.). Vocalizing such sounds may possibly induce quicker reaction times, more power, and more precise movement. This apparently intuitive and metaphorical process is attracting interest in various disciplines as a tactic in motor skills, and it is an emerging topic in sound-symbolism studies. For instance, a Japanese study of the association between auditory perceptions of consonants and bodily motion imagery hypothesizes that such association is caused by internally experienced sensorimotor representations (e.g., power, speed, precision, etc.) evoked by particular features of non-vowel phonemes, such as [m] and [p], shedding light on how proto-language may have evolved in ancestral hominids. While the Japanese study supports this hypothesis, the researchers have not yet addressed how particular phonemes are associated with aspects of proprioception or kinesthesia during the execution of movements.
Along these lines, vocalizations such as my physiotherapist’s deliberately expressive exhaling (hunnnnnnhh), which has no standard orthographic representation, and therefore may fall into nonlanguage, may display precision in their embodied context. It is worth remembering that vocalization and speaking are naturally multimodal, because human speech is situated in a body that executes the speaking. Moreover, exploring the voicing of sounds overlaps the territory of communications, sociology, and environmental understanding.
Onomatopoeia (i.e., imitative and mimetic sounds) is frequently used to express our subjective, intuitive, and sensitive feelings in daily language use, offering a link between words and sensory experience. The use of onomatopoeia varies with the language. For example, Japanese has over 1,000 onomatopoetic words, including repetitive syllables and words, which is three times more than English. It is highly common for Japanese people to use onomatopoeia to express nuance, including sounds of animals, nature, inanimate objects, feelings, and movement. I wish I could have described the pain in my right buttock to an English-speaking ER doctor when she was asking me about my symptoms. Other than using the words “sharp” or “constant”, I could not find any other apt words in English. But a Japanese doctor could have understood ギシギシ (gishi-gishi), キリキリ(kiri-kiri), ガンガン (gan-gan), グサグサ（gusa-gusa), etc.
I came across an interesting Japanese exploratory study on the use of onomatopoeia in the maintenance and improvement of knee extensor muscle strength in elderly individuals, who cannot easily check their own movements during squat training. The innovative onomatopoetic design of this home-based squat training employs automated feedback using onomatopoeia, allowing participants to self-check their movements during squatting, and to maintain motivation, with the aid of a smartphone or tablet. The idea was that onomatopoetic transformation can express differences in the movements of plural joints by describing different lengths, speeds, and angles. The word “Gu” enables Japanese people to intuitively evaluate the degree of their effort. We often use the onomatopoetic ぐうーっと (“Guuu-tsu-to”) during the execution of our own body movements, whether pulling, stretching, squeezing, pushing, or taking on weight. While the study is inconclusive, using onomatopoeia appears to be effective for expressing differences in movements of plural joints.
Another innovative use of onomatopoeia has been the development of “Onomatopen”, a novel drawing software which uses sensory-based interactive technique based on the onomatopoeia. As nearly all Japanese onomatopoeic words represent the textures of materials, the state of things, or emotions, they create a prototype which enables a user to draw various textured lines by speaking the corresponding onomatopoetic words. What a marvelous idea! While the application of the study may still be within the scope of entertainment, and potentially art therapy, others have applied onomatopoeia for understanding, modelling, and measuring the characteristics of the city of Yaizu, Japan, and its felt environment, based on city walks and voiced onomatopoeia. Using a map with a total of 533 onomatopoetic words expressed and collected by the 40 participants, it is quite fascinating to be able to explore a whole new representation of a city with a deeper sense of curiosity and connection to the place.
That study reminds me of one of my community-based projects, back in 2018, an interactive online participatory mapping using GIS, called Vancouver Wabi Sabi Map. The project aimed to enrich our sense of place and to protect the diversity of cultural ecology in Vancouver by keeping a fragile aesthetic ideology alive in whatever personal expressions were available — incomplete, impermanent, emerging, decaying, elusive, intimate, complex, or simple. We explored the fluid depths and empty spaces around us to see the beauty and life in everything. A wide range of the participants’ expressions relating to a particular place — translated visually, audibly, or by multimedia — is now preserved in the online map, waiting for others to explore our city. This year, while most of us are still staying indoors and mostly digitally connected during the pandemic, I directed my team to create a new sensory map-making self-guided program called Coming To Our Senses to find a deeper connection with our surroundings through multisensory perception. I wonder how we could use onomatopoeia to expand our expression. What sort of subculture and relationships can we possibly create and evolve together?
While feeling muscle pain in my hip and leg, I slowly stretch while vocalizing my Japanese onomatopoeia, ぐうーっと (“Guuu-tsu-to”). I am sure my muscles heard it. I give a little smile and murmur to my muscles, “I heard you and will be more careful!”