Trashscape: A portrait of A from Lingdong Huang on Vimeo.
Trashscape: A virtual environment where the user can pick up and listen to A’s trash.
Mac Version Download A Leap Controller is required to play. Mouse and 3D mouse versions are in development
Plan
I thought it was an interesting idea to know my subject and explore his mind by documenting his trash.
I planned to collect all the things my subject throws away over a period of time, along with voice recordings of the subject telling some thoughts he had in mind the moment he wanted to throw away that piece of trash.
I would then do a photogrammetric scan of all the trash, and place them in a virtual 3D world where the user can wander around, pick them up and listen to the corresponding voice recording.
Process
Trash Collection
My subject handed me a plastic bag full of all sorts of trash.
Recording My Subject
To make it as realistic as possible, we recorded the subject’s voices using an ear-shaped mic from above, so when the user listen to the trash, it sounds as if the voices are coming from the trash.
We numbered each piece of trash, from 1 to 34, and put a label on each of them. This number corresponded to the number my subject would say at the beginning of each recording, so it was impossible to confuse.
I found my subject’s speech exceed my expectation. When I was planning the project, I was thinking of the recording to be something banal such as “This is a half eaten apple. I’m throwing it away because it tastes awful.” but in fact A has something quirky and insightful to say about every piece of his trash.
Here are some sample recordings:
“Newspaper”
“Spoon Unused”
“Museum Ticket”
3D Scanning
I used photogrammetry software Agisoft PhotoScan Pro to virtualize the trash.
The software is really bad at smooth surfaces, which it cannot understand, and symmetrical objects, which it tries to warp so that the opposite sides overlap. But eventually I got 21 out of 34 trash models done. The other 13 were tiny balls of ash and hair and chips of paper that were evidently too hard for PhotoScan Pro.
The finished models really had a trash-like quality to them, which might be a problem if I was scanning any other object, but instead became a bonus since I’m scanning actual trash.
Visualization
I used Unity and Leap Motion to create the virtual world.
I imported all the trash models and voices and paired them in Unity. I programmed a virtual hand so that the user can “pick up” trash by making the gesture over the Leap controller.
A question I spent a lot of time figuring out is the environment the trash were going to situate. Using the default Unity blue sky scene certainly feels unthoughtful, yet building a customized realistic scene distracts from the trash itself. I also tried to create a void with nothing but the trash, but I felt that the idea of an explorable environment is weakened by doing so.
Finally I decided to float the trash in a black liquid under a black sky. I believe this really solved the problem, and even helped bring out the inner beauty of those trash.
Things to Improve
I’m generally happy with the result. However, there are things I need to improve.
Golan pointed out that the controls are problematic. I often face this kind of problem. Since I’m testing the software hundred of times while developing it, I inevitably train myself to really master the control. However when a new user tries to use it, they usually find it way too difficult.
I’m working on the 6DoF mouse now to make a better controlling experience.
Another problem is the hand model. Currently it’s just a plain white hand from the default Leap Motion Assets. Golan and Claire gave me a lot of ideas, such as the trash picker’s glove, tweezers, A’s hand, etc.
They also mentioned things the users can do with the trash they find, such as collecting them in a bag, sorting them, etc, which I might also implement.
I’m also thinking about improving the workflow. Currently it’s really slow, and I find myself spending hours photographing each piece of trash and struggling with crappy software to make 3d models out of them. I need to automate the whole process, so anyone can just bring in their trash and get their trashscape compiled in no time.