Interactivity Exploration

A set of sketches that explore three different interaction design theories

Semester: Semester 3
Duration: 6 weeks
Tools: Javascript, CSS/HTML, ixfx
Type: Individual
Summary: A series of sketches exploring interaction design across three approaches: deriving nuance from minimal input, shaping behavior through implicit environmental response, and enabling expressive control via gesture-based meta-manipulation.

Nuance

The material was a binary input: a single on/off button. The objective was to identify how nuance can emerge despite this limitation, without relying on richer input/output modalities. The focus was on extracting expressive variation from minimal interaction.

Interaction happens using one button. The more you play around with the square the softer it becomes

In my sketch, I implemented a deformable square controlled by repeated button presses. Initially, reaching a fully “soft” state requires sustained effort and time. With continued interaction, the system adapts: transitions become faster and require less input. The behavior models a material such as dough, which becomes more pliable through kneading. The nuance emerges from temporal accumulation and changing responsiveness rather than expanded input complexity.

Implicit Interaction

The material was a 3-axis magnetometer, and the theoretical focus was implicit interaction. The goal was to design a system that responds coherently to environmental or indirect input, emphasizing a specific aesthetic quality rather than generic responsiveness.

This sketch shows the technical setup. In this case magnetic field would attract the objects toward the magnet. Magnets with different strength produce different results

I created a group of fish-like agents that react to human presence. They exhibit a preferred flow direction, but prolonged proximity causes them to diverge and avoid the user. The interaction is indirect and continuous, shaping behavior over time rather than through explicit commands. The result is intended to feel more like an ecological response than a reactive interface.

The magnetometer was laying on the floor and the magnet was hidden in the shoe. The sketch was projected onto the floor to show how fishes react to the presence

Meta-manipulation

The material was MediaPipe’s hand pose model, with a focus on meta-manipulation: designing affordances for manipulating digital objects through embodied gestures. The emphasis was on how interaction qualities (e.g., grip, motion) influence the behavior of digital artifacts, rather than defining universal gestures.

Going from pinch to a firm grip produces different types of control over the square

In this sketch, I explored how variations in grasp influence the way an object is dragged. The inspiration came from how holding a pencil affects its behaviour. A loose pinch creates something flexible, unstable, and flimsy, while a firm grip brings rigidity and control. I tried to capture that same nuance here where the gesture shapes the quality of the drag and defines an inherent trade-off between speed and precision.

With one hand (in this case green) you throw the square and with the other (pink) you catch, and manipulate further

I also explored transferring objects between hands. Here, the user can simply release the square with a flick or a push, and it will continue moving independently across the screen. The user can initiate movement and immediately disengage, using their hand for other tasks without needing to “control” and monitor the square’s exact movement.

↑ Back to top