Interactivity Exploration

A set of sketches that explore three different interaction design theories

Year: 2024
Duration: 6 weeks
Tools: Javascript, CSS/HTML, ixfx
Type: Individual
Summary: A series of sketches exploring interaction design across three approaches: deriving nuance from minimal input, shaping behavior through implicit environmental response, and enabling expressive control via gesture-based meta-manipulation.

Context

This set of sketches explores different interaction design theories through constrained materials and behaviors.

Nuance

The material was a binary input: a single on/off button. The objective was to identify how nuance can emerge despite this limitation, without relying on richer input/output modalities. The focus was on extracting expressive variation from minimal interaction.

Interaction happens using one button. The more you play around with the square the softer it becomes.

In my sketch, I implemented a deformable square controlled by repeated button presses. Initially, reaching a fully “soft” state requires sustained effort and time. With continued interaction, the system adapts: transitions become faster and require less input. The behavior models a material such as dough, which becomes more pliable through kneading. The nuance emerges from temporal accumulation and changing responsiveness rather than expanded input complexity.

Implicit Interaction

The material was a 3-axis magnetometer, and the theoretical focus was implicit interaction. The goal was to design a system that responds coherently to environmental or indirect input, emphasizing a specific aesthetic quality rather than generic responsiveness.

This sketch shows the technical setup. In this case magnetic field would attract the objects toward the magnet. Magnets with different strength produce different results.

I created a group of fish-like agents that react to human presence. They exhibit a preferred flow direction, but prolonged proximity causes them to diverge and avoid the user. The interaction is indirect and continuous, shaping behavior over time rather than through explicit commands. The result is intended to feel more like an ecological response than a reactive interface.

The magnetometer was laying on the floor and the magnet was hidden in the shoe. The sketch was projected onto the floor to show how fishes react to the presence.

Meta-manipulation

The material was MediaPipe’s hand pose model, with a focus on meta-manipulation: designing affordances for manipulating digital objects through embodied gestures. The emphasis was on how interaction qualities (e.g., grip, motion) influence the behavior of digital artifacts, rather than defining universal gestures.

Different type of grabs produce different types of control over the square.

In my sketch, I explored variations in grasp. Inspired by holding a pencil, a loose pinch results in a flexible, unstable object, while a firm grip produces rigidity and control. The system maps grip strength and hand configuration to material behavior, creating a direct link between physical articulation and digital response.

With one hand (in this case green) you throw the square and with the other (pink) you catch, and manipulate further.

I also explored transferring objects between hands. Here, the user can simply release the square with a flick or a push, and it will continue moving independently across the screen. The user can initiate movement and immediately disengage, using their hand for other tasks without needing to “control” and monitor the square’s exact movement.

↑ Back to top