This project took quite some re-working from the start and in the end made for a very effective challenge for me with regard to learning how to navigate through a difficult backend/not well-documented issue of plumbing. It the process itself involved learning how to rip pixels from a screen and send the data over a local server into a program that would then commiunicate them to processing in a format that it could both understand, and replicate.
In the end, because I spent so much time on learning and working through the issue of trying to get the video data into processing, I had to Wizard of Oz the final interaction; seeing as how for some reason the pixel data was nit being recignized as video or photo readable by the face detecting library available to processing (OpenCV). Other than that small bug, the eye tracking data from the Tobii 4c, the video feed from Skype, and the sound, were all working perfectly well.
Beyind all of that however, I was able to successfully construct an imagined future wherein the potential for monitoring the gaze during video conferencing was realized and also allowed for a bit of play with th intervention I posed; a sort of meter that humorously rates the quality of eye contact you make with the form of the other person (and which would certainly work had it not been for the openCV issues…)
much of the the feedback I received during critique revolved around various other interventions I could include and every sense were quite helpful in generating ideas for how to continue this project if I so please. I was glad to have it looked at in this somewhat unfinished state because the quality of feedback was restricted to conceptual questions of what the world can ask in and of the world especially if existing in a context outside of class.
I created a shared new tab space where multiple people can draw and leave messages. There’s a toolbar in the top right corner. You can draw with the pencil or pen (for straight lines), erase, and change the color. As the day goes by, your drawing will scroll down out of view, leaving a clean slate for a new day. Old drawings can be viewed by scrolling down, or clicking on the left hand navigation of dates.
When thinking about telematic art, my mind goes to how we can make more intimate communication two, or a small group, of people. The new tab screen is a place we go to hundreds of times per day, and yet usually serves little purpose (aside from luring us back into our frequented sites). What if we could leave messages for each other on these screens?
Messages on a new tab don’t send you notifications, yet you can count on someone viewing it within the hour. For people apart, it’s an intersection to intentionally yet also coincidentally meet.
here’s some of my progress, please view my final project for the final result:
This is a project for Eric, but also just for me, as is most of my projects. For some context, Eric is my roommate/platonic life partner(as dubbed by tli), and I made this thinking about patterns and routine in communication and also honesty. A very embarrassing project to look at really. Also it’s mildly Undertale inspired because we played that together recently and I like it.
So the gist of it is; as we message, certain specific buttons will appear based on the current conversation. Hit one of those buttons and an message will be sent on your behalf explaining your situation/feelings. Also, because we’re both indecisive there’s a button that’ll webscrape either mine or Eric’s MyAnimeList and choose an anime one of us was planning to watch, and a button that allows you to input a list of choices for it to choose from.
- A communication tool which asks users to avoid using either the letter ‘s’ or ‘h’, and to guess who is avoiding (or not avoiding) your letter.
- I used glitch.com to make a chat room. I thought about complementary roles and how two strangers might work together (against one another) and how anonymity, both of identity and intent, can cause people to not trust or to trust each other. I thought about how regulations in communication can yield inventive solutions, and new tactics can be invented to communicate in these regulations.
- Things I wanted to implement:
- prevent spamming of guessing a player (so don’t do this)
- high scoreboard
- better UI
links to prototype glitch pages:
Infinite Cities Generator
This project is based on Italo Calvino’s book Invisible Cities, a novel which counts the tales of the travels of Marco Polo, told to the emperor Kublai Khan. “The majority of the book consists of brief prose poems describing 55 fictitious cities that are narrated by Polo, many of which can be read as parables or meditations on culture, language, time, memory, death, or the general nature of human experience.” (Thanks wikipedia)
What interested me about this novel, was how much it could be assimilated to generative storytelling and big datasets. I noticed as I read on, how closely the author was following specific rule sets, and how those same rules could be used to generate a vast amount of new stories. I was fascinated by the complexity, detail and visual quality of each city that Calvino created and decided to create more of my own.
I started by decomposing the structure of his storytelling and separated his individual texts into multiple categories, such as Title, Introduction, Qualifiers, Actions, Contradictions and Morals. I sampled actual thoughts, sentences and names from his book but also added my own to the mix. I programmed my Infinite Cities Generator in p5.js using Kate Compton’s Tracery (Thanks Kate!).
Over the course of the next few weeks, I would like to complexify my rule sets as well as create generative maps for each new city, as a way to offer a visual escape into them. In addition to that, I would like to generate pdfs and actually print the book as a way to have a physical and believable artifact by the end of the project.
Below are a couple samples of the kind of stories my Infinite Cities Generator can create:
In addition to this project, I have been working on a 3D map experience, retracing all the places I have walked to in an entire week while dealing with grief. I walk when I have things to deal with or think through, and that week I walked an average of 2h a day. I’m thinking of displaying this instead of/or in addition to the Infinite Cities Generator. It would be displayed on the LookingGlass as a 3D video playing in real time, with the camera traveling on the exact paths I did.
And in addition to THAT, I have been slaving over my thesis project Dedications I-V, a volumetric documentary on storytelling in the context of progressive memory loss. It will be taking the form of five individual chapters on memory, with five different protagonists. Although I can’t really show it just yet, this is where all of my energy has been put into.
(i’m overcompensating because i haven’t produced anything real in this class yet — whoopsie)
Android App development proved harder than I thought, and I didn’t want to phone it in, so this is very unfinished. Still, since I really like my idea and plan to continue working on it, I’ll share what I have so far.
My project is a messaging app called Rebus Chat that automatically converts typed messages to Rebus puzzles like the one below, and sends them that way.
Even the usernames and the buttons like “back” will be images rather than text. All the images will come from The Noun Project. A mock-up of the design I was thinking is below (can you guess all the messages?)
I really like this idea, because it reminds me of the puzzlehunts I love to participate in, and requires a little tricky but fun thinking to decipher each message. To convert text to Rebus, here is the pipeline I had in mind (mostly unimplemented):
- Strip out punctuation/capitalization/etc.
- Try to correct spelling errors (I am looking for an existing tool to help me do this; if I can’t find one then any “words” that don’t parse as words jump to the last bullet)
- Split everything into syllables (this will be done using Rita). We only care about pronunciation here, so “Where are you” could just as well be “Wear arr ewe”.
- For each syllable, if one interpretation of it is as a (non-abstract) noun, look it up in The Noun Project and use that picture (if multiple noun interpretations, use the most common one–there are tools for ranking words based on popularity)
- If a syllable still doesn’t have a picture, try combining it with neighboring syllables to see if it makes a two-syllable noun (if so, use that image).
- If that doesn’t work, try near rhymes.
- If there still isn’t anything, then I’m not sure what to do. Some words/phrases just aren’t 100% Rebus-able (eg. “Relevant Job Experience”–what should that even look like?). I have thought of a few options:
- Option 1: Use “word math,” like R + (elephant) for “relevant” or (smile) – S for “mile.” This seems pretty hard to do programmatically, at least robustly, and there will still be words that don’t work. Like “the.”
- Option 2: Just put those parts in as text, like “(eye) (half) the relevant job experience”. It will be as Rebus-ified as possible, and still a bit of a puzzle to decipher, but not purely images, which is too bad since I like the all-images look.
- Option 3: Just remove those parts, keeping only what can be Rebus-ified. This might turn “I have the relevant job experience” to “(eye) (half)” and then… nothing. That’s no good, because it loses important content. However, maybe in the case of just small words (a/the/and) it’s okay. This could perhaps be fused with Option 2, then?
- Option 4: Prompt the user before the message gets sent, marking that word as un-rebus-able and encouraging them to try something else. This is a little clunky and less smooth from a UI perspective, but might result in the best Rebuses.
I am leaning towards Option 2, but would be interested in hearing your opinions on this. I really do want to make this a reality, because I think it could be super fun and it really is time I learn Android App development.
My telematic project is a website where you can publish a text work and watch it get vandalized by visitors in real time. Once a work is published, each visitor is given a random word in the work that they can replace by submitting a text form. Every time a word is replaced, this is communicated to each client using sockets. The visitor also gets a new word to replace. This project is implemented in glitch.com using JQuery, Sequelize, and Socket.io.
This project’s concept has evolved far from its starting point. Originally, I was looking into text RPGs and sought to create a cairn where each visitor can add a new “page” to the text RPG. This text format naturally led me to look more into the implications of language rather than taking on a game-like structure. At that point, I fully intended to write some form of poem remixer in which visitors could do things like change the rhyme structure of a written poem or transform a word to a synonym, antonym, or other related word. This still felt lacking to me, because what was the point of this transformation? I realized that I could create a more interesting work (at least to me) by focusing on that transformative aspect. I have a long history with transformative works and the futile notion of authorship and content ownership on the internet. That, and it would be kind of funny to watch a troll vandalize your work before your very eyes.
So I opened up the format to freely change any single word in the published piece. A few limitations apply. Instead of choosing which words to change, the programming assigns you a random word. You can only replace words with single words–no spaces. In order to go to a different word, you must submit your change (or lack of change) for the current word. Additionally, the text supports HTML, which provides another avenue of mayhem in exposing the ugly syntax by getting rid of an end-bracket. Where I found the most interesting interactions to arise was in the networking. Once the work is published, the author is simply another visitor to the site. The author can see their work be vandalized in real time. Just as the vandalizers can bastardize words in the original text, the author can defend by restoring their original words or even communicate with the vandalizers by sacrificing a word with an angry, caps-locked “STOP”.