Category Archives: Capstone

Capstone Projects from IACD-2015

Matthew Kellogg – SkyComics

SkyComics is a pair of tools for drawing fully-surrounding immersive scenes. The first tool, an editor allows the user to look in all directions by dragging and zooming, and draw on their surrounding. It has tools for drawing lines and placing rectangular and circular guides. Multiple line colors can be used. The created drawing can be exported as a skybox formatted image for editing in digital painting software, and re-imported any number of times. The second tool, a viewer allows the user to view the created scene/scenes by right-clicking to progress through a list of images which can be edited in the source.

This project was created in collaboration with Caitlin Boyle. She created the scenes of room interiors. Two of these are shown below in the viewer.




The Inspiration

Caitlin’s inspiration for this project was derived from the idea of drawing an immersive hand-drawn comic that the viewer could navigate. Golan brought me this idea for me to make a tool for her to create her comic, without worry of the warping imposed by skyboxes.

What I learned

By doing this project, I was able to learn some new skills in the form of javascript and some javascript libraries. I had not yet worked with javascript and web development (outside of a few small sketches in p5.js), so this project gave me an opportunity to learn some of the pitfalls of javascript. One of these would be what some would consider an odd scoping system. Another would be the restrictions against file input and output and the complicated poorly documented HTML5 APIs which are used for file io. I had also not yet worked with ThreeJS, HammerJS, dat.gui, or JQuery. Each of these exposed me to how to use javascript libraries and how poorly documented even libraries from Google (dat.gui) can be.

What I did well

I feel that I was able to reach my goals on several aspects of the project. I feel I did a particularly good job on the view interaction. When scrolling a web-page on a touch device, users touch and drag the content. People are used to this experience. With the viewer and the editor, the user can touch and drag the surrounding environment around them. Also, in the editor pen and mouse events are separated from touch events in the editor, so the user can draw with the pen while dragging with their finger to move the scene. When no touch input is available or the user wishes to use the mouse or pen, the mouse/pen right-click can be used as touch events that are able move the scene. Also, pinching or scrolling can be used to zoom in and out.

Another thing I feel I did a good job at was the performance/efficiency. When drawing each part of a line from one frame to the next would be a separate threeJS object. As each object is iterated over for scene updates and other events, this created a huge performance problem. I created a simple system that batches strokes together in order to greatly reduce the number of objects created. This made it possible for the program to execute acceptably on both my phone and tablet.

What I didn’t do well

Initially the project I had planned for had many more features. One of these was the ability to click on regions and navigate to other scenes. I ran into problems with other parts of the project and the time I spent debugging issues far exceeded the time I thought I would spend implementing all of the planned features. Due to this project I’ve learned that I need to consider how much time it might take to debug issues or learn how to do new things.


12 May 2015


Polydues is a procedural dress that is parametrically linked to the rotations of the animation rig joints. The idea came to me while using Maya Graph Editor. I saw all the complex curves manifested by the animation and thought – “hey this would make cool data for a procedural object”. A parametric dress felt like the most interesting object for animation data since they already overlap and influence each other. The animation data is also visualized on the left and represent x, y, z absolute rotation values of each joint in the rig.

This is what the animation data looks like of a single joint.




12 May 2015


game link:

playthrough video:

A Game About Interactivity, Meaningful Choices, and Fun.

This is a game that pretends to be about a girl going through mundane activities such as playing with friends and going to school while a war deteriorates her surroundings.  The girl has to make choices between playing with her friends and helping out her family, trying to manage time and make do with what she has in a world that slowly decays over the course of 5 days.


However, none of the player’s choices matter. The girl always dies to a car bomb on day 2. But she then wakes up on day 3 with player choices stripped away because the primary means of interaction, the phone screen, is broken at the bottom. As time further progresses, the war game the player can play on the girl’s phone becomes impossible to win, and the only form of interaction the player can make is to click through a linear progression. In other words, there is no meaningful choice. In the end, even the click-through interactivity the player had is taken away, and all the player can do is watch an animation of the girl walking to and back from school unfold, ending with her laying alone in her bed.


In the end, this is not even a game by definition. There is no interactivity and no meaningful choice. Throughout this entire process, we repeatedly ask the player “are you having fun yet?”. Because it sure is not.


We made this to experiment with a decreasing rather than increasing interest curve through the mechanic of slowly removing interactivity from the game. We originally wanted to make a game about war, where the main mechanic is taking away player choices. However, we realized that our original game idea was absurd: why is the girl still going to school, even after dying? We received a lot of feedback regarding the removal of mechanics too, and based on those, we decided to remove all logic by the end to focus on showcasing the absurdity of game tropes and point out player expectations of what a “game” is supposed to be.



This game is made with Andrew Sweet and Jessica Shen using the SALD engine.


making of:


sald engine:



12 May 2015

Ol’ Scroll is a series of browser based animations, activated by user scroll. This project expands the utility of the scrollbar and uses the familiar gesture as a vehicle for an artful experience. Each animation is focused around perpetual vertical motion with the scroll bar seamlessly wrapping around the browser window.


This series was inspired by an old-time animation machine, the mutoscope. A mutoscope is a viewer driven animation machine which used a hand crank to advance frames.  Similarly, I wanted to use the user’s scroll in order to control the rate and direction of these in-browser animations.


My concerns with this project were largely formal, I wanted to emulate 3 dimensional space via color, shape and motion.


These animations were created with p5.js using a mix of SVG and computationally generated elements.

Ol’ Scroll can be viewed at my website.
More of my projects and code can be viewed on GitHub.


12 May 2015



Anyone who has worked in a large, bureaucratic organization can relate to the absurdity of Dilbert’s high-tech workplace experiences, which is why Dilbert is perhaps the most photocopied, pinned-up, and e-mailed comic strip in the world. Created by Scott Adams, Dilbert has been published for over 26 years and now appears in 2,000 newspapers worldwide.

For my final project, I’ve been working on creating the ability to create new mashups of different panels from the Dilbert comic strip based on the conversational content of each panel.

All of Dilbert’s individual comic strips can be viewed on, and, for the most part, each strip’s dialogue can be viewed in the metadata of the strip’s web page. The problem is that the transcribed dialogue on the strip’s web page isn’t separated by their individual panels.

I developed some Python code to help get around this. First, I used Beautiful Soup, a python package for parsing HTML documents, to scrape and download all of the comic strips, nearly 10,000 of them, and the strip’s transcribed dialogue. I then used the Python Image Library to take each strip and crop them into individual panels.

To get the dialogue content for each individual panel, I initially used PyTesseract, an optical character recognition tool for Python, to decipher the panel’s text content. This proved to be problematic since the text used in Dilbert is styled in a hand-written typeface, leading to recognition errors. Dilbert doesn’t use enclosed speech bubbles, so there are many cases where the speech for two different characters are spaced so closely together that the OCR tool erroneously recognizes the dialogue as a single block of text. To get around this, I used OpenFrameworks and OpenCV with a lot of help from Golan and his clever ideas to perform a series of image manipulations to isolate the character’s lines in each panel. By forming bounding boxes around each character’s dialogue, I used OCR and compared the recognized text to the transcribed dialogue, which served as the ground truth. This allowed me to assign each panel its associated dialogue.

This groundwork allowed me to create new mashups using individual panels. By using the Natural Language Toolkit and CMU’s pronouncing dictionary, for example, I created a web page that allows users to create strips that all rhyme with each other. I also used the CMU’s pronouncing dictionary to create strips that are haikus (strips consisting of 5-syllable-, 7-syllable-, 5-syllable-panels)

Screen Shot 2015-04-30 at 9.50.33 AM

This was a challenging but incredibly fun project to work on. I learned how to program in Python and learned to use data scraping methods and tools like computer vision and natural language processing.


The image manipulation operations helped a lot to recognize the dialogue, but there are still many strips that have missing or incorrectly-ordered dialogue or words. Future work can enlist workers from Mechanical Turk to read each individual panel and improve the accuracy of the transcribed dialogue that served as the ground truth for my OCR operations.

For counting syllables, there are many words in Dilbert’s dialogue (proper nouns, some corporate jargon) that are not in the CMU pronunciation dictionary. These words threw off the syllable count for generating haikus. I would like to add code to handle approximate pronunciation for these cases.

There’s much more that can be done with the individual panels and the resulting associated text. Future work can include word clouds of common phrases or words, or infographics about which characters speak to whom.

I registered a domain name to transfer my work, including 2 GB of individual comic strip panels, online, but setup issues with the server didn’t allow me to deploy this project online by today’s date. I’m still working to debug it so I can have it viewable online within the next week.

Code is available at:

All individual comic strips and separated panels are available in a single 1.6 GB file at

UPDATE: The Dilbert mashup page is viewable at