Interaction - Bachelor
Beyond the Frame is an interactive installation inspired by Edward Hopper’s Nighthawks (1942). Using hand tracking, projection and animation, the project transforms a static painting into a responsive experience. Through physical interactions with everyday objects, visitors can explore the scene and discover new ways to connect with artworks.
What if a painting could do more than sit still on a wall? What if it could respond to the person standing in front of it?
Inspired by Edward Hopper’s Nighthawks (1942), Beyond the Frame explores how interactive technology can transform the way people experience art. The project began with a simple idea: to turn a familiar painting into a space for discovery, where visitors can explore the stories and emotions hidden within the scene.
By combining physical interaction, animation and projection, the installation invites visitors to move beyond passive observation and become active participants in the artwork.
Edward Hopper’s Nighthawks depicts four figures gathered inside a brightly lit diner at night. Despite their physical proximity, the figures appear emotionally distant, creating a quiet atmosphere of isolation and ambiguity.
This sense of stillness became the starting point for the project. Rather than changing the painting completely, I wanted to explore what might happen if small details began to move and the characters’ stories became visible through interaction.
The challenge was to preserve the atmosphere and composition of the original artwork while introducing a new way for visitors to experience it.
Art Institute of Chicago. (n.d.). Nighthawks. https://www.artic.edu/artworks/111628/nighthawks
Paintings invite viewers to interpret what they see, but the experience often remains one of observation. The stories, emotions and relationships suggested by an artwork are left to the viewer’s imagination.
Beyond the Frame explores how physical interaction can encourage a more active relationship between visitors and art, without overwhelming the original work with digital effects.
The menu and newspaper become physical links between the visitor’s world and the world inside Nighthawks.
Inspired by the diner setting, these familiar objects serve as interaction points rather than separate digital controls. Visitors use hand movements detected by a webcam to activate different animated sequences, revealing new perspectives within the scene.
The menu and newspaper offer different ways to enter the painting, allowing the interaction to feel connected to the artwork’s setting and narrative.
The project developed through concept exploration, interaction design, technical prototyping and user testing.
Explore
Investigate the painting’s atmosphere, characters and potential narratives to establish a clear creative direction.
Design
Develop the menu and newspaper as physical interaction points, and storyboard how each interaction could reveal a different perspective.
Prototype
Use MediaPipe hand tracking and TouchDesigner to connect webcam input with animation playback, projection and sound.
Test
Conduct usability testing with five participants to observe how they approached the installation, understood the props and interacted with the hand-tracking system.
Refine
Adjust the left and right trigger zones, introduce a playback lock to prevent additional triggers during an animation, and remake the newspaper animation to improve its visual effect.
The experience begins with the still image of Nighthawks. As visitors interact with the menu or newspaper, the installation responds by activating animations within the painted scene.
Subtle character movements, changes in lighting and sound introduce a sense of life without replacing the original composition. Each interaction encourages visitors to look more closely and imagine the stories unfolding beyond what is immediately visible.
The aim is not to provide one definitive interpretation of the painting, but to create space for curiosity, personal interpretation and emotional connection.
Beyond the Frame explores one possible way of bringing interactive technology into the experience of viewing art. Future development could investigate additional interaction methods, clearer guidance for visitors and more responsive animation.
The concept could also be explored through other artworks, with each installation designed around the original work’s visual language, atmosphere and narrative possibilities.
Rather than applying the same effects to every painting, the goal would be to develop interactions that respond to the character of each artwork and encourage visitors to discover it in a new way.
Penelope is an Interaction Design student at QUT whose practice explores immersive experiences, interactive installations and generative art. She is interested in creating thoughtful, multisensory experiences that encourage people to slow down, connect with art, and form meaningful emotional connections with the world around them.
"*" indicates required fields