The Influence of Audiovisual Semantics on Attention

The information captured by human senses is highly structured by meaningful relationships to each other and to the scene as a whole. In visual scenes, semantic modulation of attention is well established, but had not yet been studied extensively in multisensory contexts. The work presented here establishes that audiovisual semantic relationships can guide attention in a continuous and task irrelevant manner. As described in Chapter 2, a database of semantic relatedness value was derived from human judgments to allow for a more rigorous quantification of semantic relatedness. The semantic relatedness values from this database were then leveraged in Chapter 3 to investigate how the semantic relationship between a sound and image can influence attentional priority. When the sound and target image were more strongly related, the target was found more quickly, in scale with the degree of semantic relatedness, demonstrating semantic guidance of attention in an audiovisual context. This audiovisual semantic guidance is investigated further in Chapter 4 by examining whether it depends on semantic identity to the task. With a paradigm using an orthogonal task, the data presented shows that even when both sound and image are irrelevant to the task, the semantic relationship guides attentional priority. Taken together, the work presenting in this dissertation provides strong evidence that humans have robust and continuous understanding of audiovisual semantic relatedness that modulates what information is an environment is prioritized, suggesting a larger role for semantics in audiovisual attention than previously shown.

Author

• Wegner-Clemens, Kira

Language

• en

Date created

• 2025

Type of Work

• Dissertation

License

• All rights reserved

Rights statement

• In Copyright

GW Unit

• Psychology

Degree

• Ph.D.

Advisor

• Shomstein, Sarah

Persistent URL

•  https://scholarspace.library.gwu.edu/etd/s4655h44d