Kaushal Agrawal & Eli Rosen | Project 4 | Wall Fish
Wall Fish
Wall Fish is a collaborative project between Kaushal Agrawal and Eli Rosen.
Inspiration
[vimeo=vimeo.com/1007230 width=”640″ height=”400″]
[youtube http://www.youtube.com/watch?v=IvbgAmwEX_A&width=640;height=auto;]
The Project
[youtube http://www.youtube.com/watch?v=JSWBENZBFqE&w=640&h=360]
Concept
We decided on the concept from the beginning. We envisioned a wall projected seascape that would become activated by the presence of a person. We wanted to support both passive and active engagement with the installation. Each person that moves past the installation is pursued by a school of fish. If the person is in a hurry they may not even notice their contribution to the seascape, as their fish “shadow” follows behind them. A more curious individual can engage actively with their school of fish. Standing in front of the aquarium environment allows your school of fish to catch up. They circle curiously around you but are startled by sudden changes in direction. Touching the seascape generates a small electric shock. Any nearby fish will be electrocuted. The escaping fish will flee from the electricity. Of course, fish have no memory. They’ll soon return to your side so you can shock them again.
The Technical Stuff
The wall fish installation uses processing and the Kinect to accomplish blob tracking and touch detection using OpenCV. A big thanks to Asim Mittal who helped enormously with a nice piece of code for detecting touch with the Kinect. Eli worked on creating the art, the fish behaviors, and the interactions. The behaviors and interactions were developed on the computer in response to mouse movements. Eli adapted a flocking algorithm by Daniel Shiffman. Meanwhile Kaushal developed the framework for tracking multiple people’s movement across the installation and for detecting a touch. He used Shiffman’s openKinect library. Kaushal then adapted Eli’s code, replacing mouse movement with blob tracking, and replacing the mouse click with a touch on the projected display.
Challenges
One of the challenges was detecting touch on a wall surface with a skewed perspective and needed calibration with the Kinect and the Projection. We decided not to use OpenNI for detection of people so it was difficult to track multiple people in cases of collision and overlap, so we ended up using probabilistic detection of blobs nearest to the previous blob position.
Trying to accomplish an interesting set of behaviors for the school of fish was also a challenge. Working only with human position and speed, we wanted to create a personality for the fish that was both curious and tentative. In order to create this effect we experimented with a number of behaviors and interactions.
Optical flow might be a useful technique for having the actions of the user affect the fish flocking.
Fish should avoid the shadow/silhouette … they shouldn’t be projected on the back. maybe. i like that it seems a seperate fishtank on the wall.
Fish that might actually survive a college dormroom. Shocking.
fun diversity of interactions, attract/repell/surprise. great video showing non-creator interactions.
Cool video and soundtrack. Very experimental feeling, which I think works. +1
+ Agreed. If you could show in the video 10 people playing, that would enhance the perception of this interaction as engaging and fun. I know it’s hard though :)
Not sure that is in the kinect scope – but would love to see the ability to introduce objects and have the game interact with them. Like a nurf gun, or throwing food for the fish.. so they would flock around it. Just on top of my head.
I’m impressed by how polished you made the project and how elaborate the setup is. Seems like a great start on what could easily turn into a final, seems like you learned a lot about doing larger installation-like projects.
yay fishies! looks like a lot of fun to play with, especially the electricity. the graphics definitely help make the piece. ^_^>
This is very easy to be understood hope to play with it. ANd the visuals are great.
Great work combining your individual talents! And nice job solving the physical challenge of tracking.
I like the depth of interaction that you can do just more than have the fish follow you, but also manipulate them to an extent as well.
I really like how the swarms follow people accross the screen. It’s simple but effective.
Great shot of people smiling after shocking the fish – really sells the project.
Good documentation and attention to detail.
+1 The blog post seems really nicely detailed
^Definitely! Good to see such a firm commitment to the idea