Meg Richards – Project 2 Final

by Meg Richards @ 3:14 pm 4 February 2011

Network Usage Bubbles

The original incarnation of this project was inspired by the Good Morning! Twitter visualization created by Jer Thorp. A demonstration of CMU network traffic flows, it would show causal links for the available snapshot of the network traffic. All internal IP addresses had to be anonymized, making the internal traffic less meaningful. Focusing only on traffic with an endpoint outside of CMU was interesting, but distribution tended towards obeying the law of large numbers, albeit with a probability density function that favored Pittsburgh.

This forced me to consider what made network traffic interesting and valuable, and I settled on collecting my own HTTP traffic in real-time using tcpdump. I summarily rejected HTTPS traffic in order to be able to analyze the packet contents, from which I could extract the host, content type, and content length. Represented appropriately, those three items can provide an excellent picture of personal web traffic.

Implementation

The visualization has two major components: Collection and representation. Collection is performed by a bash script that calls tcpdump and passes the output to sed and awk for parsing. Parsed data is inserted into a mysql database. Representation is done by Processing and the mysql and jbox2d libraries for it.

Visualization Choices

Each bubble is a single burst of inbound traffic, e.g. html, css, javascript, or image file. The size of the bubble is a function of the content size, in order to demonstrate the relative amount of tube it takes up to other site elements. Visiting a low-bandwidth site multiple times will increase the number of bubbles and thus the overall area of its bubbles will approach and potentially overcome the area of bubbles representing fewer visits to a bandwidth-intensive site. The bubbles are labeled by host and colored by the top level domain of the host. In retrospect, a better coloring scheme would have been the content type of the traffic. Bubble proximity to the center is directly proportional to how recently the element was fetched; elements decay as they approach the outer edge.

The example above shows site visits to www.cs.cmu.edu, www.facebook.com (and by extension, static.ak.fbcdn.net), www.activitiesboard.org, and finally www.cmu.edu, in that order.

Network Bubbles in Action

Code Snippets

Drawing Circles

Create a circle in the middle of the canvas (offset by a little bit of jitter on the x&y axes) for a radius that’s a function of the content length.

Body new_body = physics.createCircle(width/2+i, height/2+j,sqrt(msql.getInt(5)/PI) );
new_body.setUserData(msql.getString(4));

Host Label

If the radius of the circle is sufficiently large, label it with the hostname.

if (radius>7.0f) {
    textFont(metaBold, 15); 
    text((String)body.getUserData(),wpoint.x-radius/2,wpoint.y);
  }

tcpdump Processing

Feeding tcpdump input into sed

#!/bin/bash
tcpdump -i wlan0 -An 'tcp port 80' | 
while read line
do
if [[ `echo $line |sed -n '/Host:/p'` ]]; then 
    activehost=`echo $line | awk '{print $2}' | strings`
...
fi

The full source

Project 2: The Globe

by huaishup @ 6:30 am 2 February 2011

1. Overall
When talking about data visualization, most of the people will think of computer graphic visualization. However, from my view, this is only one of the possible ways to do it. Why not trying visualizing data in physical ways? People can not only see the visualization result, but can also touch and manipulate the visualization device, which could be really interesting.

In this project, I explores the physical/tangible way of visualizing data. Using a paper globe as the data media, people can learn the language of a certain area by spinning the globe and adjusting the probe.

2. Material

Arduino x1

WaveShield with SD card x1

Button x1

Speaker x1

Variable resister x2

Paper

Wood

Wire

3. Process

a. Prepare the paper globe
Using google images to download one LANGE size global map. Download a Photoshop plugin called Flexify 2 to revise the map images. Here is the tutorial. Plot the revised image, cut and glue.

b. Fix the variable resistor
Laser cut 4 pieces of round woods to fix the shape of the paper globe. May use extra timber to do so. Install one of the variable resistor to the bottom of the globe. See below.

c. Install all the other parts
Install another variable resistor as the probe which points to the globe. Lazer cut a seat for the probe and the globe. Hook up two different Analog Input pins with the Arduino and resistors.

d. Calculate the position
Spin the globe and alter the probe. Different position has different resistor value, which can be mapped to the sound track. Calculate the position and map the sound track.

e. Prepare the sound
Download the language sound from Google translate and store them in the waveshield.

4. Video

5. Code

#include 
#include 
#include 
#include "WaveUtil.h"
#include "WaveHC.h"
 
char decision = 0;
SdReader card;    // This object holds the information for the card
FatVolume vol;    // This holds the information for the partition on the card
FatReader root;   // This holds the information for the filesystem on the card
FatReader f;      // This holds the information for the file we're play
 
WaveHC wave;      // This is the only wave (audio) object, since we will only play one at a time
 
#define DEBOUNCE 100  // button debouncer
 
int mySwitch = 7;
 
// this handy function will return the number of bytes currently free in RAM, great for debugging!
int freeRam(void)
{
  extern int  __bss_end;
  extern int  *__brkval;
  int free_memory;
  if((int)__brkval == 0) {
    free_memory = ((int)&free_memory) - ((int)&__bss_end);
  }
  else {
    free_memory = ((int)&free_memory) - ((int)__brkval);
  }
  return free_memory;
} 
 
void sdErrorCheck(void)
{
  if (!card.errorCode()) return;
  putstring("\n\rSD I/O error: ");
  Serial.print(card.errorCode(), HEX);
  putstring(", ");
  Serial.println(card.errorData(), HEX);
  while(1);
}
 
void setup() {
  // set up serial port
  Serial.begin(9600);
  putstring_nl("WaveHC with 6 buttons");
 
  pinMode(mySwitch,INPUT);
 
   putstring("Free RAM: ");       // This can help with debugging, running out of RAM is bad
  Serial.println(freeRam());      // if this is under 150 bytes it may spell trouble!
 
  //  if (!card.init(true)) { //play with 4 MHz spi if 8MHz isn't working for you
  if (!card.init()) {         //play with 8 MHz spi (default faster!)
    putstring_nl("Card init. failed!");  // Something went wrong, lets print out why
    sdErrorCheck();
    while(1);                            // then 'halt' - do nothing!
  }
 
  // enable optimize read - some cards may timeout. Disable if you're having problems
  card.partialBlockRead(true);
 
// Now we will look for a FAT partition!
  uint8_t part;
  for (part = 0; part < 5; part++) {     // we have up to 5 slots to look in
    if (vol.init(card, part))
      break;                             // we found one, lets bail
  }
  if (part == 5) {                       // if we ended up not finding one  :(
    putstring_nl("No valid FAT partition!");
    sdErrorCheck();      // Something went wrong, lets print out why
    while(1);                            // then 'halt' - do nothing!
  }
 
  // Lets tell the user about what we found
  putstring("Using partition ");
  Serial.print(part, DEC);
  putstring(", type is FAT");
  Serial.println(vol.fatType(),DEC);     // FAT16 or FAT32?
 
  // Try to open the root directory
  if (!root.openRoot(vol)) {
    putstring_nl("Can't open root dir!"); // Something went wrong,
    while(1);                             // then 'halt' - do nothing!
  }
 
  // Whew! We got past the tough parts.
  putstring_nl("Ready!");
}
 
void loop() {
  //putstring(".");    // uncomment this to see if the loop isnt running
  if(digitalRead(mySwitch) == HIGH) {
    Serial.println("switch is ok");
    int spin = analogRead(5);
    int probe = analogRead(2);
    Serial.println(spin);
    Serial.println(probe);
 
    if(spin>=0 && spin<=576 && probe >=179 && probe <=276) {
      playcomplete("r.wav");
    }
    else if(spin>=85 && spin<=313 && probe >=35 && probe <=160) {
      playcomplete("c.wav");
    }
    else if(spin>=580 && spin<=780&& probe >=0 && probe <=142) {
      playcomplete("a.wav");
    }
    else if(spin>=980 && spin<=1023 && probe >=7 && probe <=22) {
      playcomplete("p.wav");
    }
    else if(spin>=980 && spin<=1023 && probe >=0 && probe <=7) {
      playcomplete("s.wav");
    }
    else if(spin>=1023 && probe >=47 && probe <=288) {
      playcomplete("e.wav");
    }
    delay(1000);
  }
 
}
 
// Plays a full file from beginning to end with no pause.
void playcomplete(char *name) {
  // call our helper to find and play this name
  playfile(name);
  while (wave.isplaying) {
  // do nothing while its playing
  }
  // now its done playing
}
 
void playfile(char *name) {
  // see if the wave object is currently doing something
  if (wave.isplaying) {// already playing something, so stop it!
    wave.stop(); // stop it
  }
  // look in the root directory and open the file
  if (!f.open(root, name)) {
    putstring("Couldn't open file "); Serial.print(name); return;
  }
  // OK read the file and turn it into a wave object
  if (!wave.create(f)) {
    putstring_nl("Not a valid WAV"); return;
  }
 
  // ok time to play! start playback
  wave.play();
}

shoosein_project2 regrets

by shoosein @ 4:14 am

This is a description of the game I should have made for the Info Viz project.

The goal is to build a path through a maze from the starting point to the end point by placing facts within grid squares. The player will need to read their tiles carefully to ensure they don’t place a lie, because if they do their path will be reset and they will have to rebuild it from the beginning again. Once complete, the tiles that contain facts that support the topic question will be filled with one color while those that contain facts that contradict the topic question will be filled with another. The player’s bias would be illustrated by a majority of tiles of one color over the other.

The maze will be designed so that the player does not need to use every fact at their disposal and at least one route has an even number of squares. The even number would allow the player to remain unbiased if they truly have no opinion on the topic question posed, but the lesser number of facts needed would give their bias plenty of opportunity to show without impacting the efficiency of their solution. The puzzle-ness of the game means that there’s no need for a timer, so unlike my initial version the player would be able to read each fact at their own pace.

Diagram of a possible solution, the S=start point, star=finish, F=fact text.

Maya Irvine – Project 2 – Final Presentation

by mirvine @ 7:06 pm 1 February 2011

The role of the internet in relationships is a topic of great interest to me. Applications such as gchat and skype present an entirely new forum for interaction that has likely had a major affect on the way we conduct ourselves romantically.

With this and several technical goals in mind, I decided to investigate the role that one program, gchat, played in one of my previous relationships.

I drew a lot of inspiration from XKCD for this project. Many of this comics installments have featured the protagonist mathematically analysing a relationship.

This is meant to be absurd of course, but they touch on the very real fact that reflection and analysis is a major part of relationships. Ironically, as Andy Warhol suggests in the following quote, nobody is every truly aware of what is going on in a relationship while it is taking place, a problem he chose to solve with the documentation provided by a tape recorder.

“I didn’t get married until 1964 when I got my first tape recorder. My wife, my tape recorder and I have been married for ten years now.”

So my central questions were these:
What can my gchat record tell me about my relationship? How did we use this program? and how does that written record compare with my memory of what happened?

This is what resulted:

A scatterplot of all chats. Periods of increased and decreased communication are clearly visible. Also preferred times of day.

School breaks (time apart) start to explain clusters of chats.

Seperating out the speakers showed a relative equality in our overall amount of chats.

Emoticons are a large part of chatting. They clarify the ambiguity of text, but do they reveal anything about its true sentiment?

<3

smilies of all kinds. Interestingly, I found that both happy and sad smilies denote positive conversations.

x’s

laughter. The prevalence of laughter in these conversations surprised me.

the words “love you”

Conclusions:
Ultimately, I have to admit that while interesting, this info graphic will never be as revealing to anyone else as it is to me. I can make comparisons to my own memories that no one else can make, because I was there. Further more, this viz is not so much representative of a relationship as it is of how that relationship was conducted. Wich of course only leads to the question, are the two separate? Either way, there is a greater truth here that is not being tapped.

The most revealing part of the viz was the scarcity of the phrase “love you”. In comparison to all the other searches, this phrase is not a clarifier, but one that holds meaning in itself. The fact that it was so rare reveals the lack of actual sentiment being expressed in the rest of the chats.

Code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
//Project2 - Data Viz 
//<3<3<3
//Maya Irvine
//Thanks goes to Mickey Reiss, Samia Ahmed, Matt Sandler, Mauricio Giraldo
//Interactive Art and Computational Design Spring 2011
//Carnegie Mellon University
 
BufferedReader reader;
String line;
Conversation currentConversation;
Message currentMessage;
MessageLine currentMessageLine;
ArrayList<Conversation> convos;
 
PFont gothamRounded12;
PFont gothamRounded50;
 
String [] months = {" ","dec","jan","feb","mar","apr","may","jun","jul","aug","sep","oct","nov","dec", "jan"};
String [] hours = {"12"," "," ","3"," "," ","6"," "," ","9"," ", " ","12"," "," ","3"," "," ","6"," "," ","9"," "," "};
 
int topMargin, breakValue, roll;
color pink;
 
 
void setup()
{
  size(1000, 800);
  background (255);
  noStroke();
  smooth();
  parse();  
  yAxis();
  xAxis();
 
  title();
 
  topMargin = 150;
  breakValue = 255;
  color pink = color(201, 2, 75);
  roll = 0;
 
 
}
 
void draw() 
{ 
  buttons();
 
  breaks();
 
 
 
 
      gothamRounded12 = loadFont("GothamRounded-Bold-12.vlw"); 
      textFont(gothamRounded12, 12); 
 
     for (int i = 0; i < convos.size(); i++) { 
      Conversation conversationI = ((Conversation)convos.get(i));
      //print( conversationI.month+" ");
      //print( conversationI.day+" ");
      //println( conversationI.year); 
 
 
 
 
       for (int j = 0; j < conversationI.messages.size(); j++) { 
         Message messageJ = ((Message)conversationI.messages.get(j));
            //print( messageJ.hour+":");
            //print( messageJ.minutes+" ");
            //println( messageJ.meridien);
 
 
 
            for (int k = 0; k < messageJ.messagelines.size(); k++) { 
              MessageLine messagelineK = ((MessageLine)messageJ.messagelines.get(0));
              //print( messagelineK.speaker+" : ");
              //println(messagelineK.words);
 
 
 
                 if(messagelineK.speaker.equals("me"))
                 {
 
                   //fill(255);
                   fill(201, 2, 75, 30);
                 }
                  if(messagelineK.speaker.equals("Paul"))
                 {
                   noStroke();
                   //noFill();
                   fill(0, 107, 198, 30);
                 }
 
//                  String[]temp = match(messagelineK.words, "x\\b|xx\\b|xxx\\b|X\\b|XX\\b|\\:x\\b");
//                  if (temp != null) {
//                    
//                   noStroke();
//                   fill(252,240,0);
//                 }
//                 
//                 String[]temp4 = match(messagelineK.words, "love you");
//                   if (temp4 != null) {
//                     noStroke();
//                     fill(252,240,0);
//                     }
 
//                     String[]temp4 = match(messagelineK.words, "\\bha|\\bhaha|\\bhe|\\bhehe|\\blol");
//                   if (temp4 != null) {
//                     noStroke();
//                     fill(252,240,0);
//                     }
//                     
//                     String[]temp5 = match(messagelineK.words, "\\<3");
//                   if (temp5 != null) {
//                     noStroke();
//                     fill(252,240,0);
//                     }
//                 
//                 else
//                 {
//                   noStroke();
//                   fill(0, 25);
//                 }
 
//                 String[]temp2 = match(messagelineK.words, "\\:\\)|\\:D|\\:P|\\;\\)|\\;D|\\;P|xP");
//                  if (temp2 != null) {
//                    
//                   noStroke();
//                   fill(252,240,0);
//                  }
//                  
//                  String[]temp3 = match(messagelineK.words, "\\:\\(|\\:\\'\\(");
//                  if (temp3 != null) {
//                    
//                   noStroke();
//                   fill(252,240,0);
//                  }
 
 
 
            if(messageJ.meridien.equals("AM"))
            {
 
             ellipse(((conversationI.year-9.85)*(365*2))+(60*conversationI.month)+(conversationI.day*2),topMargin+((messageJ.hour)*20)+messageJ.minutes/3, 15,15);
            }
 
            if(messageJ.meridien.equals("PM"))
            {
              if(messageJ.hour == 12)
             {
               messageJ.hour = 0;
             }
             ellipse(((conversationI.year-9.85)*(365*2))+(60*conversationI.month)+(conversationI.day*2),(topMargin+(12*20))+(messageJ.hour*20)+messageJ.minutes/3, 15,15);
            }
 
 
 
            noLoop(); 
 
    }
       }
     }
}
 
 
 
 
void yAxis()
{
  for (int a = 0; a < hours.length ; a++)
  {
    gothamRounded12 = loadFont("GothamRounded-Bold-12.vlw"); 
      textFont(gothamRounded12, 12); 
    fill(0, 150);
    text(hours[a], width/20, 150+(a*20)+20);
 
  }
}
 
void xAxis()
{
  for (int b = 0; b < months.length ; b++)
  {
    gothamRounded12 = loadFont("GothamRounded-Bold-12.vlw"); 
      textFont(gothamRounded12, 12); 
    fill(0, 150);
    text(months[b], (10-9.85)*(365*2)+b*60-60,700);
 
  }
}
 
 
 
 
void breaks()
{
  if(mouseX > (width*4/15-10) && 
     mouseX < (width*4/15+20) && 
     mouseY > height*1/7-20 && 
     mouseY < height*1/7+20)
 
           {  
              breakValue = 0;
              roll = pink;
              println("clicked");
           }
 
  rectMode(CORNERS);
  fill(0,40);
  rect((10-9.85)*(365*2)+3*60+4*2, 160, (10-9.85)*(365*2)+3*60+14*2,660);
  rect((10-9.85)*(365*2)+12*60+13*2, 160, (10-9.85)*(365*2)+13*60+9*2, 660);
  rect((10-9.85)*(365*2)+1*60+13*2-60, 160, (10-9.85)*(365*2)+1*60+11*2, 660);
  rect((10-9.85)*(365*2)+5*60+11*2, 160, (10-9.85)*(365*2)+8*60+23*2, 660);
 
}
 
void title()
{
  gothamRounded50 = loadFont("GothamRounded-Bold-50.vlw"); 
  textFont(gothamRounded50, 50);
  fill(0); 
  text("<3 <3 <3", width/2-120, height/10);
}
 
void buttons()
{
  gothamRounded12 = loadFont("GothamRounded-Bold-12.vlw"); 
  textFont(gothamRounded12, 12); 
  fill(0);
  text("me", width*4/15,height*1/7);
  fill(0);
  text("you", width*5/15,height*1/7);
 
  text("<3", width*6/15,height*1/7);
  fill(0);
  text(":)", width*7/15,height*1/7);
 
  text("x", width*8/15,height*1/7);
  fill(0);
  text("ha", width*9/15,height*1/7);
  fill(0);
  text("love", width*10/15,height*1/7);
  fill(0,30);
  text("breaks",width*11/15,height*1/7);
}
 
 
 
 
 
 
void parse()
{
  convos = new ArrayList<Conversation>(); 
  reader = createReader("chats.txt");
 
  while(true) {
    try {
      line = reader.readLine();
    } catch (IOException e) {
        e.printStackTrace();
        line = null;
      }  
 
    if (line == null) {
    // Stop reading because of an error or file is empty
      break;
    } else {
     // println (line);
      String[] newConversation = match(line, "\\*\\*\\*\\*\\*");
      if(newConversation != null) 
        {
        //Create a new conversation
        currentConversation = new Conversation();
        convos.add(currentConversation);
        }
      if(line.equals("")) 
        {
        continue;
        }
 
      String[] date = match(line, "show details (\\d{1,2})/(\\d{1,2})/(\\d{2,4})");
    if(date != null) {
      //println("Found a year "+date[3]);
       //println("Found a day "+date[2]);
       //println("Found a month "+date[1]);
      currentConversation.year = Integer.parseInt(date[3]);
      currentConversation.month = Integer.parseInt(date[1]);
      currentConversation.day = Integer.parseInt(date[2]);
    }
 
    String[] wordsAndSpeaker = match(line, "(Paul|me): (.*)");
    if(wordsAndSpeaker != null) {
      currentMessageLine = new MessageLine();
      currentMessage.messagelines.add(currentMessageLine);
      //println("Found a message "+wordsAndSpeaker[2]);
      //println("Found a speaker "+wordsAndSpeaker[1]);
      currentMessageLine.speaker = (wordsAndSpeaker[1]);
      currentMessageLine.words = (wordsAndSpeaker[2]);
 
    }
 
    String[] time = match(line, "(\\d{1,2}):(\\d{1,2}) (AM|PM)");
    if(time != null) {
      currentMessage = new Message();
      currentConversation.messages.add(currentMessage);
     // println("Found an hour "+time[1]);
      //println("Found minutes "+time[2]);
      //println("Found meridien "+time[3]);
 
      currentMessage.hour = Integer.parseInt(time[1]);
      currentMessage.minutes = Integer.parseInt(time[2]);
   //  currentMessage.setMeridien(time[3]);
      currentMessage.meridien = (time[3]);
     // println(currentMessage.meridien);
 
    }
 
 
  }
    }
}
class MessageLine
{
  String speaker;
  String words;
}
 
class Message 
{
  int hour;
  int minutes;
  String meridien;
  ArrayList<MessageLine> messagelines;
 
 public Message() 
  {
    messagelines = new ArrayList<MessageLine>();
  }
}
 
class Conversation 
{
  int month; 
  int day; 
  int year;
  ArrayList<Message> messages;
 
  public Conversation() 
  {
    messages = new ArrayList<Message>();
  }
}

Maya Irvine – project2 – progress report

by mirvine @ 6:34 pm

This progress report is a bit late, but hopefully it will illustrate some of my process.

“I didn’t get married until 1964 when I got my first tape recorder. My wife, my tape recorder and I have been married for ten years now.”

After scraping the data, my immediate thought was to create a timeline and scatter plot to see if there were any trends over time. I first wanted to do my entire chat history, mapping everyone I had ever spoken with online, but after considering time, I felt that it would be more practical and meaningful to focus on one relationship. Here are some early sketches thinking about orientation:

Next I started messing about with the data, and realizing how much I needed to learn in order to do this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
//import java.util.Date;
//import java.text.SimpleDateFormat;
 
void setup()
{
  size(300, 200);
  background (255);
  noStroke();
 
 
}
 
void draw()
{	
  date();
}
 
void date() {
 
String chats[] = loadStrings("chats.txt");
for (int i=0; i < chats.length; i++) 
{
  String[] m1 = match(chats[i], "\\*\\*\\*\\*\\*");
    if (m1 != null) {
 
//       SimpleDateFormat dateformatMMDDYYYY = new SimpleDateFormat("MM/dd/yyyy");
//  text(dateformatMMDDYYYY.format(chats[i+3]), width-100,35);
 
      }
 
      String[] m2 = match(chats[i], "AM");
    if (m2 != null) {
       println(chats[i]);
 
      }
 
        String[] m3 = match(chats[i], "PM");
    if (m3 != null) {
       println(chats[i]);
 
      }
 
 
 
 
 
 
 
 
    String[] m4 = match(chats[i], "Paul\\:");
    if (m4 != null) {
      // This will print to the console.
      //println("Found a match in '" + s1 + "'");  
      fill(255, 0, 0);
      ellipse(i, height/2, 2, 2);
    }
 
    String[] m5 = match(chats[i], "me");
    if (m5 != null) {
      fill(0, 0, 255);
      ellipse(i, height/2, 2, 2);
      //println("Found a match in '" + s2 + "'");
    }  
 
 noLoop();
}
 
 
 
}

This was my first output. Not very meaningful, but a start.

I needed a system for sorting and storing the different parts of the txt file I wanted to work with. Lots of diagrams of arrays were drawn with encouragement from Mickey Reiss.

After succeeding in creating the structure, I had to figure out how to navigate through it. This is me trying to figure out the syntax with help from Samia.

Alex Wolfe | Data Visualization

by Alex Wolfe @ 8:34 am 31 January 2011

Beginning

I’m personally terrified of falling. I used to be a big rock climber, and one time I was sort of shimmying my way up a chimney, it’s this narrow space and there are no handholds so your back is wedged up against one wall and your feet the other and you just try to walk your way up. But I was too short and my shoes were too slippery and a lost my grip, my baleyer wasn’t paying attention so I just fell. I was pretty high up and it was probably only 100ft before  he stopped the rope and grabbed me, but it felt like eons, and I was so scared and I kept thinking of that unpleasant wet *thwack* sound I’d make when I finally hit the bottom.

So I have a sort of morbid fascination for people who’d jump of their own free will. Initially when I started this project I had this idea of flight vs. fall, visually displaying all the people who jump each year, and showing who survived and who didn’t seeing as I myself came so close to the statistic. I really wanted to highlight the falling motion, and probably the dramatic splat I’d so envisioned.

Data

I stumbled across the 21st century mortality dataset which was this comprehensive list of everyone who’d died since 2001 in england, and exactly where and how they died. It was ridiculously huge, with over 62,000 entries, each storing multiple deats. They used the ICD-10 International Classification of Diseases which is brutally specific to categorize them. Just looking for deaths related to falls earthed up 17 different categories, ranging from meeting your demise by leaping off of a burning building to death by falling off the toilet. However, when I went digging around for survivors, there wasn’t anything half so comprehensive. BASE jumpers are assigned a number when they complete all 4 tasks, skydiving companies keep some vague handwavy statistics, and I found several lists of people who’d died trying. However those crazy people who jump for fun typically are up to some crazy(illegal) stunts, such as underwater base jumping, or walking out to the edge of a wind turbine so there is no easy way to count/find/categorize them all with half the level of detail as the less fortunate.

So i decided to focus on the dataset I had. I wrote a quick javascript that essentially just trolled through the dataset, which was stored as a .cvs file, and pulled out any deaths filed under codes related to falling and put them in a nice new .cvs file

First Stab/Brainstorming

Since I had that jumping/falling effect in mind, I went through and made each person I had into his/her own particle. Mass/Radius I based on the age of the person who died, color based on gender, and I stored any information other information about them in the particle. I put some basic physics and started playing around. I had this idea where I could simulate the “jump” with each particle starting from the height of the person’s final leap, and I could hand-draw a graphic for the side displaying the more common places.

Here was my initial attempt

Final

Although interesting, it wasn’t particularly informative at all, so i abandoned the “jumping effect” and focused on other things I could get the particles to do. Ultimatly I executed blobbing based on gender, and then sorting themselves into the ICD-10 categories of death, keeping hold of the “falling effect” during the in between transitions. I wanted to have each stage look like the particles just fell into place create the visualization

Critique

Although I love the freeform effect of the falling particles, and their transition from displaying each of the patterns, it doesn’t really do the data justice. I have so many juicy details stored in there I just couldn’t display. With the number of particles, it was horribly slow if you did mouseover display, and each one was virtually unique as far as age, place, cause of death, gender, so there weren’t any overarching trends for the really interesting stuff. I think I’m going to go back and guess my best estimate for height, hardcode it in and maybe do a state where it attempts to display it, or at least a few more things, eg. line up by age and mouseover explain ICD-10 code. I really want to think of a way to get the cause of death to be a prominent feature for each individual

Chong Han Chua | App Store Visualization | 31 January 2011

by Chong Han Chua @ 8:31 am

This project explores the possibility of doing an interesting visualization of icons from the Apple iTunes App Store.

This project was conceived when I saw the visualization of flags by colours. The whole nature of a set of similar manner graphics, such as  the flags, amused me immensely. It then suddenly occurred to me that the icons on the iTunes App Store are of the same nature. Almost rectangle, with rounded corners, and usually vector graphics of some sort – would be interesting to look at.

I guess in a way, this existed as largely a technical inquiry. This is the first time I wrote a crawler as well as a screen scraper. This is the first time I dealt with a large set of data that takes almost forever to do anything with. I can almost feel that heart beat when I ran the scraping script for the first time, half expecting Apple to boot me off their servers after a few hundred continuous queries. Thankfully, they didn’t.

There are a bunch of technical challenges in this inquiry mainly:

1. Scraping large sets of data requires planning. My scraping code went through at least 3 different versions, not to mention playing with various language. Originally, I wanted to use Scala as I was under the impression that the JVM would be more efficient as well as speedy. Unfortunately, the HTML returned by the iTunes App store is malformed – one of the link tags is not properly closed and choked the built in Scala’s XML parser.

After determining that using any random Java XML parser would be too much of a hassle, I turned to my favourite scripting language, JavaScript on node.js (using Google V8). After looking through a bunch of DOM selection solutions, I finally got jsdom and jquery to work, then I knew that I was in business.

The original plan was to crawl the website from first page to last page and create a Database entry for every page in the website. There was only very basic crash recovery in the script which basically state that the last scraped entry is a certain index n. Unfortunately for me, the links are traversed not exactly in the same order every time so I ended up having duplicate entries in my database. Also, the script was largely single threaded, and it took almost over 10 hours to scrape 70+k worth of pictures.

After realizing that a partial data set will not do me any good, I decided to reconcentrate my efforts. I then built in some redundancy in getting links and test the data base for existing entries before inserting. I also ran another script on top of the scraper script that restarts the script when it crashes on a bad response. Furthermore, I used 20 processes instead of 1 to expedite the process. I was half expecting to really get booted off this time round, or get a warning letter from CMU but thankfully till now there is none. After 10 hours or so, I managed to collect 300,014 images. Finder certainly isn’t very happy about that.

2. Working with large data sets requires planning. Overall, this is a pretty simple visualization, however the scaffolding required to process the data consumes plenty of time. For one, there was a need to cache results so that it doesn’t take forever to debug anything. SQLite was immensely useful in this process. Working of large sets of data also means that when there is a long running script, and it crashes, most of the time, the mid point data is corrupted and has to be deleted. I pretty much ran through every iteration at least 2 to 3 times. I’m quite sure most of my data is in fact accurate, but the fact that a small portion of the data was corrupted (I think > 20 images) does not escape me.

I wouldn’t consider this a very successful inquiry. Technically it is ego stroking, on an intellectual-art level, there seems to be no very useful results from the data visualization. When constructing this visualization, I had a few goals in mind

1. I don’t want to reduce all these rich data sets into simply aggregations of colours or statistics. I want to display the richness of the dataset.

2. I want to show the vastness of the data set.

As a result, I ended up with a pseudo spectrum display of every primary colour of every icon in the App Store that I scraped. It showed basically the primary colour distribution in something that looks like a HSB palette. The result was that it seems to be obvious that there are plenty of whitish or blackish icons, and the hue distribution of the middle saturation seems quite even. In fact, it says nothing at all. It’s just nice to look at.

There’s a couple of technical critiques on this: The 3D to 2D mapping algorithm sucks. What I used was a very simple binning and sorting via both the x and y axis. Due to the binning, the hue distribution was not equal for all bins. To further improve this visualization, the first step is to at least equalize the hue distribution across bins.

I guess what I really wanted to do, if I had the time, was to have a bunch of controls that filters the icons that showed up on the screen. I really wanted to have a control where I can have a timeline where I can drag the slide across time and see the appstore icons populate or a bunch of checkboxes which I can show and hide categories that the apps belong to. If I have more chops, I would attempt to sort the data into histograms for prices and ratings and have some sort of colour sorting algorithms. If I had more chops, I would make them animate from one to another.

I think there is a certain difficulty in working with big data sets as there is no expectation of what trends to occur since statistically speaking everything basically evens out and on the other hand it basically just takes forever to get anything done. But it is fun, and satisfying.

If you want the data set, email me at johncch at cmu dot edu.

Code can be found at:

Scraper
Data prep
Visualization

Project 2-Infoviz-Nisha Kurani – Final

by nkurani @ 8:13 am

Project Focus

What makes a movie a hit?  Do people watch movies with strong plots?  Is there a trend in which genre people enjoy watching?  I’ve always assumed that box office hits are always the movies with the most amazing plots.  Why else would so many people spend their money to watch these movies?  Why would some people watch them multiple times?

Finding the data

For as long as I could remember, I’ve been consulting IMDB (http://www.imdb.com/) to see how people have ranked a movie I’m about to watch.  Many times I use the site to guide my decision.  I found a list of the Top 474 Box Office Hits; however, when I went to download the data, I realized that it was going to be very challenging.  After struggling with it, I decided to find another source.  I ended up using a list from Box Office Mojo (http://boxofficemojo.com/alltime/domestic.htm) and extracted the data from the site into 4 text files.  Each file included 100 of the top 474 all-time box office hits.  In the end, I decided to focus on the top 100 since it was a more manageable data set.  I felt using more than 100 would clutter the screen.

While searching for more data, I brainstormed other data that I could provide in addition to the information from the Box Office Mojo site that provided me with each movie’s rank, title, studio, lifetime gross, and year. I fiddled with the possibilities of including the movie’s genre, rating, director, release date, lead actors and more.  To find the corresponding information for each movie, I had to do some digging.  I searched for an IMDB API which lead me to their “Alternative Interfaces.”  If you’re interested in the data, I acquired my set from: ftp://ftp.sunet.se/pub/tv+movies/imdb/


Scraping the data


Scraping the data took the majority of my time.  I ended up downloading really big list files that included ratings/genre data from tv shows to international movies!  The files I used (genres.list and ratings.list) were very messy.  It was challenging, but I eventually extracted the data and saved them into text files that I could access more conveniently later.  What was even more frustrating was figuring out ways to display the information.  New to processing, I always had to do some research before executing my idea.

While figuring out ways to scrape the data, I found a few data visualizations about box office hits that I found useful.  Here they are:

Displaying the Data

I decided to plot the ratings on the y-axis and timeline on the x-axis.  I would then increase/decrease the size of the circle depending on how much money that movie made.  I also changed the color of the circle based on the movie’s genre.  In the end, I found it a useful chart that would provide its viewer with a quick way to view trends in box office movies.  I found it interesting that not all the highest ranking movies would get the best ratings.

I’ve included my code in a zip file in case you’d like to take a look at it.  Please let me know if you have any questions or suggestions regarding my infoviz.  I look forward to making some of the changes that were discussed in class.

Below is a screenshot of the latest representation of my application.  When you roll over the buttons, it displays the movie name.  When you click on a bubble, you get additional information about the movie including: box office gross, year, genre, rating, and the number of votes it received.

REFLECTION

After the critique, I realized I should have tested out more variations of displaying the data in processing instead of just sketching and disregarding things.  Also, I could make the visualization stronger by providing information like release dates, directors, and a “star factor” that would list the number of big stars in the movie.  I plan on exploring these areas in the weeks to come.  Since the critique, I’ve redone the colors to make the differences in genre more visible and labeled my axes.  I tried to make it so that when you scroll over the genre key to the right of the page, you only see the bubbles that are part of that genre; however, the code wasn’t functioning correctly so I decided to leave it out for now.

This assignment was really fun.  It was the first time I’ve ever created a visualization with code.  I found it challenging at times, but it was definitely worth it.  Over the course of this project, I’ve developed a long list of visualizations I’d like to create, so I’m excited to test those out in the weeks to come.

TO DOWNLOAD CODE: http://nishakurani.com/infoViz/myMovieViz.zip


SamiaAhmed-Final-Infoviz

by Samia @ 7:15 am

For my project, I wanted to explore the data of my life. For fall and spring semester of my sophomore year, I kept a detailed, fairly complete log of all of my actions in an analogue notebook. I wanted to see if there were any connections I could draw out of my everyday actions.

One of the biggest problems I ran into was simply getting the data into digital form. I had (and still have) to type it up personally because no one else can read my handwriting or understand the shorthand.

After I had a few days of data written, and a working parser, I began to run the data with a first, basic visualization of a pie chart. I mapped events catagorized as sleep to a dark purple, school to blue, fun to pink, housekeeping to yellow, and wasting time to green. In the screen shots below, I also gave them a transparency with alpha. NOTE: because I am silly, these graphs start midnight at 0300, and then move through the day clockwise.

the image below is of maybe 3 or 4 days worth of data. Already patterns are emerging — a desaturated area where I sleep, a yellow/olive band of waking up and getting up in the morning. two blue peaks of morning and afternoon classes, and a late afternoon pink band of doing fun after class stuff.

by this point, probably around 10 days of data, the patterns are pretty clear — especially the sharp line between black and yellow when I wake up every morning.

the pie chart re-oriented so that 0000 hours is midnight. 0600 is midday.

below I looked at the data sequentially, each day as a horizontal band.

My final interaction looked like the below images: each of the bands is a day of the week. So monday, for example, is the three days of semi-transparent monday data graphed on top of one another. patterns are pretty obvious here – the front of the school week has lots of blue for homework and school. The afternoons of thursday and friday are fairly homework free, etc

Clicking on a day allows you to compare the “typical” day with specific ones, as well as compare the events broken down by catagory (how many hours of school work vs that days’ average)

All in all, I’m glad I got this to work in some capacity. I think the data would be more interesting if I had all of it. In terms of interaction and design there are lots of weak points — poor labelling, jarring color-coding, and non-intuitive buttons.

For concept however, Golan hit the nail on the head. As I was transcribing the data, I was really excited to work with some of the specific things I tracked — for example, when I tried to get up in the morning and failed, or when I did my laundry, or how much time I spent doing homework, verses in class, what times of day I biked. I think I was so caught up in getting the “overview” of the project to work that I never got to those more interesting and telling points. In retrospect, my time may have been better spent digitizing the data about, perhaps, when I slept, and then just working with that, since it became obvious that I would not have time to put in the entire database. A smaller subset of the information might have conveyed a more understandable picture — for example seeing that I’m biking home from campus at 2 in the morning might just as well convey I had a lot of work to do as writing all the tasks of that day.

Caitlin Boyle :: Project 2 InfoViz

by Caitlin Boyle @ 6:35 am



My idea came from…, various exercises in frustration. In a way, the hardest part about this project was just committing to an idea… Once my initial project fell through, my attack plan fell to pieces. I’m not used to having to think in terms of data, and I think I got too wrapped up in the implications of “data”. Really, data could have been anything… I could have taken pictures of my belongings and made a color map, or done the same for my clothing; but in my head, at the time, I had a very specific idea of what the dataset I was searching for was, what it meant, and what I was going to do once I found it. I think stumbling upon the ruins of the government’s bat database put me in too narrow a mindset for the rest of the project… for a week after I realized the batdata wasn’t going to fly, I went looking for other population datasets without really questioning why, or looking at the problem another way. It took me a little longer than it should have to come back around to movie subtitles, and I had to start looking at the data before I had any idea of what I wanted to visualize with it. My eventual idea stemmed out of the fluctuation of word frequency in different genres; what can you infer about the genre’s maturity level, overarching plot, and tone by looking at a word map? Can anything really be taken from dialogue, or is everything in the visuals? The idea was poked along with thanks to Golan Levin and two of his demos; subtitle parsing and word clouds in processing.

Data was obtained… after I scraped it by hand from www.imdb.com ‘s 50 Best/Worst charts for the genres Horror, Comedy, Action and Drama. .srt files were also downloaded by hand because I am a glutton for menial tasks I’m a novice programmer, and was uncomfortable poking my nose into scripting. I just wanted to focus on getting my program to perform semi-correctly.

Along the way, I realized… how crucial it is to come to a decision about content MUCH EARLIER to open up plenty of time for debugging, and how much I have still to learn about Processing. I used a hashtable for the first time, got better acquainted with classes, and realized how excruciatingly slow I am as a programmer. In terms of the dataset itself, I was fascinated by the paths that words like “brother, mother, father” and words like “fucking” took across different genres. Comedy returns a lot of family terms in high frequency, but barely uses expletives; letting us know that movies that excel in lewd humor (Judd Apatow flicks, Scary Movie, etc.) are not necessarily very popular on imdb. On the other hand, the most recurring word in drama is “fucking”, letting us know right away that the dialogue in this genre is fueled by anger.

All in all I think I gave myself too little time to answer the question I wanted to answer. I am taking today’s critique into consideration and adding a few things to my project overnight; my filter was inadequate, leaving the results muddied and inconclusive. I don’t think you can get too much out of my project in terms of specific trending; the charm is in it’s wiki-like linking from genre-cloud, to movie titles, to movie cloud, to movie titles, to movie cloud, for as long as you want to sit there and click through it. I really personally enjoy making little connections between different films that may not be apparent at first.

Subtitle Infoviz ver. 1 video

Pre-Critique Project

Post-Critique (coming soon) :: more screenshots/video/zip coming soon… making slight adjustments in response to critique, implementing labels and color, being more comprehensive when filtering out more common words. I plan to polish this project on my own time.

Project 2: Data Visualization – Mapping Our Intangible Connection to Music

by Asa Foster @ 4:28 am

General Concept

Music is an incredible trigger for human emotion. We use it for its specific emotional function a lot of the time, using music to cheer us up or calm us down, as a powerful contextual device in theater and film, and for the worship of our deities of choice. Although it is very easy for an average listener to make objective observations about tempo and level of intensity, it is harder to standardize responses to the more intangible scale of how we connect to the music emotionally. This study aims to gain some insight on that connection by forcing participants to convert those intangible emotional responses to a basic scale-of-1-to-10 input.

The goal of this project is to establish a completely open-ended set of guidelines for the participant in order to collect a completely open-ended set of data. Whether correlations in that data can be made (or whether any inference can be made based on those correlations) becomes somewhat irrelevant due to the oversimplification and sheer arbitrariness of the data.

Execution

An example of an application of a real-time system for audience analysis is the response graph at the bottom of the CNN screen during political debates. The reaction of the audience members, displayed by partisanship, is graphed to show the topic-by-topic approval level during the speech. By having a participant listen to a specific piece of music (in this case, Sufjan Stevens’ five-part piece Impossible Soul) and follow along using a program I created in Max/MSP to graph response over time, I can fashion a crude visual map of where the music took that person emotionally.

Data & Analysis

Data was gathered from a total of ten participants, and the graphs show some interesting connections. First off are the similarities within the opening movement of the piece; from talking with the participants there seemed to be a general sense of difficulty standardizing one’s own responses. This led to a general downward curve once the listener realized that there was a lot more breadth to the piece than the quiet opening lets on. Second is the somewhat obvious conclusion that the sweeping climax of the piece put everyone more or less towards the top of the spectrum. The third pattern is more interesting to consider: people were split down the middle with how to approach the song’s ending. To some it served as an appropriately minimalist conclusion to a very maximalist piece of music, to others it seemed forced and dry.

Areas of Difficulty & Learning Experiences

  • The song is 25 minutes long, far too long for most CMU students to remove their noses from their books.
  • As the original plan was to have a physical knob for the listener to use, I had an Arduino rig all set up to input to my patch when I fried my knob component and had to scale back to an on-screen knob. Nowhere near as cool.
  • A good bit of knowledge was exchanged for the brutal amount of time wasted on my initial attempt to do this using Processing.
  • I have become extremely familiar with the coll object in Max, a tool I was previously unaware of and that has proved EXTREMELY useful and necessary.

Code

Download Max patches as .zip: DataVis

Emily Schwartzman – InfoViz – Color of the News

by ecschwar @ 4:27 am

The Question
This exploration began with the question of what would be an interesting data set. I considered several sources and decided to work with data from Jonathan Harris’s 10×10. Every hour of every day 100 words and pictures are collected from leading news sources around the world. The data, kindly made available for artists and developers to work with, was neatly organized and stored by date, with all images saved at a consistent size.

Process
After selecting data, my initial thought was to do something that compared or integrated the images and words together. After seeing some examples of other information visualization pieces in class, I was inspired to look at the color of the news. My focus question that persisted throughout this project was “What is the color of the news?” I sketched a few ideas for how this could be visualized. Some ideas that emerged were to create a circle for each year that would give an impression of the dominant colors. Expanding on this, I decided to go with a tree ring structure, with rings expanding out from the starting year. This would allow for comparisons within each ring to look at individual years, as well as the opportunity to make comparisons across years by month.

Inspiration
I was inspired by Flickr Flow, by Fernando Viegas and Martin Wattenberg, which I discovered after beginning my project. This piece visualized the colors of flickr images over the course of a year, and is both beautiful and insightful. I saw some similarities with my project in both the content and form. When I was looking for Processing examples for how to execute the visualization, I found an example from toxiclibs called Polar Unravel, which mapped strokes of color on a polar coordinate system. This is what inspired the idea to represent each day of the year as a stroke of color on the circle.

The Data
Thanks to Mauricio, who shared a PHP script with me that I could modify, the data was scraped from 10×10 over the course of about a week. After slowly running in the background on my computer for many days, I had around 450,000 images saved on my computer. I’ve never had so many images in one folder before. Needless to say, Finder did not respond well. Prepping the data from these images was the most challenging and time-consuming part of the project. I started with a toxiclibs demo, ImageColors in the colorutils library, to identify the dominant colors from each image. A color histogram is created from each image, identifying colors that appear most frequently. A tolerance variable controls how similar the colors extracted will be. The higher the tolerance, the fewer the number of colors returned on the histogram. I also had some examples from Ben Fry that used a Float Table class to map data from a .tsv file. I modified the ImageColors code to output the r,g and b values, frequency and date information into a text file, which I then saved as a tsv file to use in the visualization component. I began working with a sample of around 9,000 images to start testing on. I went back and forth between working on the code to create the tsv file and the actual visualization. I was able to pull dominant colors from individual images, but given that there were 200 images for each day I needed to minimize the dominant color to one or several for each day. I struggled the most with this component. Thanks to some technical help from Ben, I was able to get the color histogram to run twice, once on individual images and once on an array of color values extracted from individual images for each day. This output one table with a specified number of colors for each day, based on the frequency of that color. When I got to the point of experimenting with data from all of the images, I had to cut the number of images in half to make it more manageable, so instead of 200 images a day I reduced it to 100 images per day.

Visualizing Process
Below are some screenshots of my process as I was playing around with different ways of visualizing the color. The early shots were working with several weeks worth of data. I experimented with a few days of showing the color on a ring. My initial idea was to look at one dominant color for each day, but ultimately decided that several colors would give a better impression. I went with 6 colors for each day, stacking 3 colors per stroke. I did not see any immediately visible patterns emerging in the results from years worth of data, so I wanted to try another way of looking at it. Instead of doing each ring as a year, I also looked at doing each ring as a month, so one circle would represent a year. However, because of the way the colors are structured, the density of each ring was significantly greater in the middle rings, so the comparisons were not consistant enough from ring to ring.

Final Visualization
Below is the end result. (download full size)

Reflection
Overall I am happy with the end result. I was a little disappointed that there was no immediate pattern visible, but this could be because of the variety of images used in the news. A lot of brownish colors were also visible, which could be from all of the people in the images. As mentioned in the critique, the method used to extract the dominant image could be improved. The colors would vary slightly each time the code was run, so I know that there are still some issues to be worked out. I had originally wanted to work with the 100 words data that was collected, but did not have the time to look at that for this first study. I still think it would be interesting to compare what the dominant colors for each day were with the top words for each day. Another idea I wanted to explore was to indicate when significant events occurred, to see if the events influenced the color of the news for any length of time. I also question the ring form. While I find it effective from a visual standpoint, I think there may be other ways to communicate the information.

Code
Download Processing files

JamesMulholland-Project2-InfoViz_geneHack

by James Mulholland @ 4:17 am

For my information visualization project I chose to dive into my own family tree. The data comes from a file assembled by an estranged aunt (whom I’ve never met) She converted to the Mormon church, which has a strong mission of researching family lineage.

Hoping to add insight about family longevity, the areas of color represent four major eras in European history with different life-expectancies. The saturation of the color relates how long each person lived. Persons depicted in gray do not have enough data to determine lifespan. Individuals are placed on the y-axis according to their birth-date. For those without a birth-date, I had to crudely estimate the date based on family relations.

Using Processing I parsed a GEDCOM genealogy file, organized the data using a hashmap,  and generated the tree structure using recursion. In Processing, rolling over an individual reveals the name and birth-year.

PROCESS

Using JGenealogy I could export a kml document to view the geographical distribution by generation. Mildly interesting, but the map search was a little inaccurate. The geography data available pointed to England mostly as well as northern Europe and a couple different areas in the US (especially midwest and east coast). In JGeneaology and GEDitCOM II, I was able to view the data and browse and edit. But with so many people, I couldn’t view them all at once and still get an idea of the information.

I knew previously that going back far enough I could trace my line to Henry Plantagenet (Henry II of England, ruled 1154–1189AD). The fact itself is pretty exciting so I wanted to see that relationship. William FitzNorman (of the Normandy de la Mere clan, b1048) is the oldest on my tree. However since he was a descendant of Rollo the Viking (Duke of Normandy) there are records that go back much further… perhaps even to Fjornjot “the Wind” of Finnish folklore(b 120 AD)… !?

Since my family intends to expand and enhance the data over time, I thought it would be great to have a basis for interacting with the data in a more interesting way, perhaps even to be used as a tool for identifying a browsing through parts of the data. Now unfortunately, this information is just what is known from my grandpa’s side of the family. I don’t have very extensive information on the other branches of ancestry.

About the Data:
The data is in a common genealogy GEDCOM format used by genealogy software.  There are almost 80000 lines in the file, between 12-18 lines per entry for 4900 entries. Below is an example:

0 @I1608@ INDI
1 NAME John /Alling/
2 SURN Alling
2 GIVN John
1 SEX M
1 BIRT
2 DATE 2 Oct 1647
2 PLAC New Haven, New Haven, Connecticut
1 DEAT
2 DATE 25 Mar 1717
2 PLAC New Haven, New Haven, Connecticut
1 _UID 854F56280FD63E4BBE99447EDF1F60BEF526
1 FAMS @F758@
1 FAMC @F761@
1 CHAN
2 DATE 9 Jun 2002
3 TIME 15:35:17

The dataset itself caused me the most grief because there were many behaviors of the software and characterstics of the data that I didn’t realize until much later. Some entries included date of birth, some included date of death. I also made an 11th hour realization that some parents were parents of more than one family, so some of the relationships are not drawn as extensively as they should be.

After parsing, I set up a HashMap to store all of the family ID numbers and then drew the tree using recursion. Given that I have very little experience with parsing or hashmaps, I was pleased to have been able to utilize them in this project. Recursion was also new to me, but I gained some experience understanding how not to avoid too much recursion.

I used to think I was almost completely Irish and German. But it’s an odd feeling looking back so far where the English and Scandinavian and French all blend together. Can this composition be quantified like on the NY Times site? Perhaps that’s on tap for the next round.

Process sketches:

Code available on Pastebin.

Presentation (from Prezi):

shoosein_Project 2

by shoosein @ 3:49 am

So I made a game for the information visualization assignment. Facts that are pro or cons to the set topic float down from the top of the screen and the player tries to collect as many as possible. At the end, a bar graph shows how many cons vs pros they collected and the facts they missed. In theory, it’s supposed to be a fun way to visualize/measure the player’s bias.

Except it completely fails to measure player bias since it’s way too easy to collect all the facts, and even the ones missed are missed due to lack of timing and have nothing to do with what the facts say. I should have put more time into research and chosen a better topic, as the “facts” are disgustingly general statements derived from studies conducted by “experts” that are really just bullshit with numbers. Worst of all, the game is not fun. It sucks. Major fail.

So in the 5 hours remaining before class starts I’m attempting to redesign and recode this. But this is what I have to show if I can’t make it in time. Enjoy. Except you won’t. Because it sucks.

And on top of it all the minim library is causing problems, so I had to comment out the awesome music my friend Christopher Vu volunteered to make for it. Watch it be something obvious.

Susan Lin — InfoViz, Final

by susanlin @ 3:14 am

Visualizing a Flaming Thread
InfoViz based off of WSJ article “Why Chinese Mothers are Superior” comments thread.

This is a looong post, so here’s a ToC:

  • The Static Visualization
  • Everything Presented Monday
  • Process Beginnings
  • Pitfalls
  • Retrospective

The Static Visualization

As per the advice during the critique, I create a infographic based off the static variant of my project. I decided to keep the 10×100 grid layout for aesthetic reasons (not having 997 bubbles all in one row and thus making a huge horizontal graphic).

This alternative version of the above offers the same thing with the areas of interest highlighted.

Everything Presented Monday
Links for everything pre-critique.

Process Beginnings

Like mentioned, the data source of interest was this WSJ Article. The article sparked some serious debate leading to threads such as these. Here is a particularly powerful answer from the Quora thread from an anon user:

Drawing from personal experience, the reason why I don’t feel this works is because I’ve seen an outcome that Amy Chua, the author fails to address or perhaps has yet to experience.

My big sister was what I used to jealously call “every Asian parent’s wet dream come true” [… shortened for conciseness …]
Her life summed up in one paragraph above.

Her death summed up in one paragraph below.
Committed suicide a month after her wedding at the age of 30 after hiding her depression for 2 years.

I thought the discussion around it, though full of flaming, was very rich with people on both ends of the spectrum chiming in. My original idea was to take apart the arguments and assemble it in a form which would really bring out the impact, similar to the excerpt from Quora.

I started off with the idea of having two growing+shrinking bubbles “battle.” More information can be read on this previous post.

This was the baseline visual I devised:

  • Green and Orange balls collide with each other.
  • Collision: green does not affect green, likewise, orange did not affect orange.
  • Colliding with opposition shortens your life span (indicated by opacity).
  • Touching an ally ups your life span.

Giving credit where credit is due:
The code started with Bouncy Balls and was inspired by Lava Lamp.

Next, I wanted to see if I could work some words into the piece. Word Cloud was an inspiration point. In the final, I ended up using this as the means of picking out the words which charged comments usually contained: parent, Chinese, and children.

Cleaning up the data:

  • When I downloaded the RSS feed of the comments, it was all in one line of  HTML (goody!).
  • With some help, I learned how to construct a Python script to organize it.
  • Basically, the code figures out where each time stamp and comment is relative to the mark-up patterns, and separates the one line out to many lines.
import re
 
f = open('comments.html', 'r')
 
text = ''
for line in f:
 
    while 1:
        m = re.search('#comment.*?#comment\d+', line)
        if m is None:
            break
 
        comment = line[:m.span()[1]]
        n = comment.find("GMT") + 4
        text += comment[:n] + "\n"
        text += comment[n:] + "\n\n"
 
        line = line[m.span()[1]:]
f.close()
 
f2 = open('comments-formatted.html', 'w')
f2.write(text)
f2.close()

Sources: comments.html, comments-formatted.html

More looking outwards:

While working, I often took a break by browsing Things Organized Neatly. It served both as motivation, inspiration, and admittedly procrastination. Also, if I could revise my idea, maybe something interesting to look at in a future project would be commonly used ingredients in recipes (inspired by above photo taken from the blog).

Pitfalls

The greatest downer of this project was discovering that language processing was actually quite hard for a novice coder to handle. Here were abandoned possibilities, due to lack of coding prowess:

  • LingPipe Sentiment Analysis – This would have been really freaking cool to adapt this movie review polarity to a ‘comment polarity’ analysis, but unfortunately, this stuff was way over my head.
  • Synesketch – Probably would have been a cool animation, but didn’t get to show two emotions at once like the original idea desired.
  • Stanford NLP – Again, admired this stuff, but way over my head.

Retrospect
In no order, some of the things I learned and discovered while doing this project.

  • Language processing is still a new-ish field, meaning, it was hard to find a layman explanation and adaptation. It would have been nice to do more sophisticated language processing on these comments, but language processing is a monster on its own to tackle.
  • Vim is impressive. I now adore Vim users. (Learned during the Python script HTML clean-up portion of this project.)
  • Mechanical Turk: This might have been an alternative after figuring out language processing was hard to wrangle. Though building a framework to harvest this data is unfamiliar territory as well (probably with its own set of challenges).
  • Another field: I really wanted to map this variable out, especially after harvesting it, but the time stamp was not used. An animation with the time stamps normalized by comment frequency may have added another layer of interpretation. Addition: Though, from the critique, it seems like more layers would actually hurt more than help. Still, I wonder if in the static visualization the time stamp could have added.
  • All-in-all: I thought this was parsed down to the simplest project for 2+ weeks… This clearly wasn’t the case. Lesson: Start stupidly simple next time!
  • As for things that went well: I forced myself to start coding things other than simple mark-up again, which is very pleasing when things come together and start working.
  • I am pleased with the combined chaos+order the project exudes (lava lamp on steroids?). The animation made for a poor visualization compared to the static version even though I spent 80% of my time getting the animation to work. On the bright side, I would have never found out without trying, so next time things will be different.

Charles Doomany- InfoVis: Final Post

by cdoomany @ 2:29 am

Digital Flora

This project acquires realtime environmental data (ambient light and temperature) from several distinct geographic locations and uses the data as a parameter for driving the recursive growth of a virtual tree. Each tree serves as a visual indicator of the environmental conditions of their respective geographic location. When the optimal conditions are met for plant growth (~7000 lumens/ 18.3 °C) the animation displays a fully matured tree at its last stage of recursion.

I used Pachube to acquire the data and Processing to generate the tree animation.

Ideas for Improvement:

• Add more parameters for influencing growth ( ex: daily rainfall, soil pH, etc.)

• Increase the resolution of growth (currently only ten levels of recursive depth)

• Growth variation is not observable over short periods of time, but is only apparent over long term seasonal environmental changes

• Current animation appears fairly static, there is an opportunity to add more dynamic and transient animated elements that correspond with environmental conditions

• An ideal version of the program would have multiple instances of the animation running simultaneously, this would make it possible to compare environmental data from various geographic locations easily

• A viewable history of the realtime animation would be an interesting feature  for accessing and observing environmental patterns

• More experience with recursively generated form and some aspects of OOP would certainly have helped me reach my initial goal

Timothy Sherman – Project 2 – ESRB tag cloud

by Timothy Sherman @ 1:17 am

My project is a dynamic tag cloud of word frequency in video game titles. The user can select a number of ratings and/or content descriptors (short phrases that describe the content of a game), assigned by the Entertainment Software Ratings Board (ESRB), and the cloud will regenerate based on the narrower search. The user can also hover their mouse over a word in the cloud, and access a list of the games with the given rating/descriptor parameters which contain that word in their title.

Initially, my data source was going to be a combination of the Entertainment Software Ratings Board’s rating data against sales numbers from VGChartz. After getting my data from both websites using scrapers (both XML based in Ruby and Nokogiri, but one that loaded 35000 pages and took 5 hours to run), I encountered a problem. The ESRB and VGchartz data didn’t line up – titles were listed differently, or co-listed in one, and listed separately in the other. There were thousands of issues, most unique, and the only way to fix it would be by hand, something I didn’t have time or patience for. I decided to drop the VGchartz data and just work with the ESRB data, as it seemed more relatable on it’s own.

Though I had my data, I didn’t really know how to visualize it. After a lot of coding, I ended up with what basically amounted to a search engine. You could search by name, and parametrize it with ratings or content descriptors, and recieve a list of games that matched. But this wasn’t a visualization! This was basically what the ESRB had on their site. I felt like I had hit a wall. I’ve never done data visualization work before, and I realized that I hadn’t thought about what to actually do with the data – I’d just thought about the data’s potential and assumed it’d fall into place. After thinking about it, I came up with a couple potential ideas, and I decided the first one I’d try would be a word frequency visualization on the game titles, one that could be parametrized by content descriptors and rating. This was what ended up being my final project.

I was working in Processing, using the ControlP5 library for my buttons, and Golan’s Tag Cloud demo which used OpenCloud. I began by coding the basic functionality into a simple and clean layout – I ended up liking this layout so much that I only modified it slightly for the final project. The tag cloud was easy to set up, and the content-descriptor-parametrized search wasn’t terrible either. I added a smaller number next to each word, showing how many times that word appeared in the search, to help contextualize the information for the viewer. I saw that there was some interesting stuff to be found, but wanted more functionality. What I had gave no information about the actual games that it used to make the tag cloud. When I saw an odd word in a given search, I wanted to be able to see what games had that name in their title. I added a scrollable list that pops up when the user mouses over a word in the cloud which lists all the games in the search with that word.

At this point, most of my work became refining the parameters I wanted to allow users to search, and various visual design tasks. I figured out colors and a font, added the ability to search by a games rating, and selected the parameters that seemed more interesting.

Overall, I’m decently happy with the project. It’s the first data visualization I’ve ever done, and while I feel that it shows to some extent, I think that what I came up with can be used to find interesting information, and there are some unintuitive discoveries to be made. I do feel that had I been thinking about how I would visualize my data earlier, I would’ve been able to achieve a more ambitious or refined project – there are still some problems with this one. The pop-up menus, while mostly functional, aren’t ideal. If you try to use them for words in small font, they become unscrollable. I had to compromise on showing the whole title in them as well. There was no way to make it fit and still display a lot of information in the table, and no way to make the table larger and still keep track of if the mouse had moved to another word – limitations of ControlP5 which I didn’t have time to figure out how to get around. That said, these are issues with a secondary layer of the project, and I think the core tag cloud for chosen descriptors is interesting and solid.

Presentation Slides

Processing Source:
Note: This requires the ControlP5 library to be installed.

import controlP5.*;
 
ControlP5 controlP5;
 
int listCnt = 0;
 
Button[] ratingButt;
Button[] descripButt;
 
Button[] textButt;
boolean changeTextButt = true;
ListBox hoverList;
int listExists = -1;
 
color defaultbg = color(0);
color defaultfg = color(80);
color defaultactive = color(255,0,0);
color bgcolor = color(255,255,255);
color transparent = color(255,255,255,0);
color buttbgcolor = color(200,0,0);
color buttfgcolor = color(150);
 
Cloud  cloud;
 
float  maxWordDisplaySize = 46.0;
 
int combLength;
String names[];
String rating[];
String descriptors[];
ArrayList descripSearch;
ArrayList rateSearch;
String descriptorList[];
String ratingList[];
ArrayList currentSearch;
PFont font;
ControlFont cfont;
 
void setup() 
{
  font = createFont("Helvetica", 32, true);
  cfont = new ControlFont(font);
  cfont.setSize(10);
  //cfont.setColor();
  textFont(font, 32);
  size(800, 600);
  smooth();
  background(bgcolor);
  frameRate(30);
  cloud = new Cloud(); // create cloud
  cloud.setMaxWeight(maxWordDisplaySize); // max font size
  cloud.setMaxTagsToDisplay (130);  
  controlP5 = new ControlP5(this);
  controlP5.setControlFont(cfont);
 
  //rating list
  ratingList = new String[5];
  ratingList[0] = "Early Childhood";
  ratingList[1] = "Everyone";
  ratingList[2] = "Teen";
  ratingList[3] = "Mature";
  ratingList[4] = "Adults Only";
 
 
  //rating buttons
  ratingButt = new Button[5];
  for(int i = 0; i < 5; i++)
  {
    ratingButt[i] = controlP5.addButton("rating-"+i,i,(10+(0)*60),40+i*24,104,20);
    ratingButt[i].setLabel(ratingList[i]);
    ratingButt[i].setColorBackground(defaultbg);
    ratingButt[i].setColorForeground(defaultfg);
    ratingButt[i].setColorActive(defaultactive);
  }
 
 
 
  //descriptor list - used with buttons for faster lookup.
  descriptorList = new String[17];
  descriptorList[0] = "Tobacco";
  descriptorList[1] = "Alcohol";
  descriptorList[2] = "Drug";
  descriptorList[3] = "Violence";
  descriptorList[4] = "Blood";
  descriptorList[5] = "Gore";
  descriptorList[6] = "Language";
  descriptorList[7] = "Gambling";
  descriptorList[8] = "Mild";
  descriptorList[9] = "Realistic";
  descriptorList[10] = "Fantasy";
  descriptorList[11] = "Animated";
  descriptorList[12] = "Sexual";
  descriptorList[13] = "Nudity";
  descriptorList[14] = "Comic Mischief";
  descriptorList[15] = "Mature Humor";
  descriptorList[16] = "Edutainment";
 
  //descrip buttons
  descripButt = new Button[17];
  for(int i = 0; i < 17; i++)
  {
    descripButt[i] = controlP5.addButton("descrip-"+i,i,(10+(0)*60),180+(i)*24,104,20);
    descripButt[i].setLabel(descriptorList[i]);
    descripButt[i].setColorBackground(defaultbg);
    descripButt[i].setColorForeground(defaultfg);
    descripButt[i].setColorActive(defaultactive);
  }
 
  //load strings from file.
  String combine[] = loadStrings("reratings.txt");
  combine = sort(combine);
  combLength = combine.length;
  names = new String[combLength];
  rating = new String[combLength];
  descriptors = new String[combLength];
  descripSearch = new ArrayList();
  rateSearch = new ArrayList();
  currentSearch = new ArrayList();
 
 
  //this for loop reads in all the data and puts into arrays indexed by number.
  for(int i = 0; i < combLength; i++)
  {    
    //this code is for the ratings.txt file
    String nextGame[] = combine[i].split("=");
    names[i] = nextGame[0];
    rating[i] = nextGame[2];
    descriptors[i] = nextGame[3];
    currentSearch.add(names[i]);
    listCnt++;
    String nameWords[] = split(names[i], " ");
    for(int z = 0; z < nameWords.length;z++)
    {
      String aWord = nameWords[z];
      while (aWord.endsWith(".") || aWord.endsWith(",") || aWord.endsWith("!") || aWord.endsWith("?")|| aWord.endsWith(":") || aWord.endsWith(")")) {
        aWord = aWord.substring(0, aWord.length()-1);
      }
       while (aWord.startsWith(".") || aWord.startsWith(",") || aWord.startsWith("!") || aWord.startsWith("?")|| aWord.startsWith(":") || aWord.startsWith("(")) {
       aWord = aWord.substring(1, aWord.length());
       }
      aWord = aWord.toLowerCase();
      if(aWord.length() > 2 && !(aWord.equals("of")) && !(aWord.equals("and")) && !(aWord.equals("the")) && !(aWord.equals("game")) && !(aWord.equals("games"))) {
        cloud.addTag(new Tag(aWord));
      }
    }
  }
}
 
void controlEvent(ControlEvent theEvent) {
  // with every control event triggered, we check
  // the named-id of a controller. if the named-id
  // starts with 'button', the ControlEvent - actually
  // the value of the button - will be forwarded to
  // function checkButton() below.
  if(theEvent.name().startsWith("rating")) {
    ratingButton(theEvent.controller());
    search(0);
  }
  else if(theEvent.name().startsWith("descrip")) {
    descripButton(theEvent.controller());
    search(0);
  }
 
}
 
void descripButton(Controller theCont) {
  int desVal = int(theCont.value());
  int desInd = descripSearch.indexOf(descriptorList[desVal]);
  if(desInd == -1)
  {
    descripSearch.add(descriptorList[desVal]);
    theCont.setColorBackground(buttbgcolor);
    theCont.setColorForeground(buttfgcolor);
  }
  else
  {
    descripSearch.remove(desInd);
    theCont.setColorBackground(defaultbg);
    theCont.setColorForeground(defaultfg);
  }
}
 
void ratingButton(Controller theCont) {
  int ratVal = int(theCont.value());
  int ratInd = rateSearch.indexOf(ratingList[ratVal]);
  if(ratInd == -1)
  {
    rateSearch.add(ratingList[ratVal]);
    theCont.setColorBackground(buttbgcolor);
    theCont.setColorForeground(buttfgcolor);
  }
  else
  {
    rateSearch.remove(ratInd);
    theCont.setColorBackground(defaultbg);
    theCont.setColorForeground(defaultfg);
  }
}
 
void draw()
{
  background(bgcolor);
  textSize(12);
  fill(255,0,0);
  text(listCnt,45-textWidth(str(listCnt)),30);
  fill(0);
  text("/"+combLength+" games",45,30);
  //text("games games",20,30);
  List tags = cloud.tags();
  int nTags = tags.size();
  // Sort the tags in reverse order of size.
  tags = cloud.tags(new Tag.ScoreComparatorDesc());
  if(changeTextButt)
  {
    textButt = new Button[130];
  }
  float xMargin = 130;
  float ySpacing = 40;
  float xPos = xMargin; // initial x position
  float yPos = 60;      // initial y position
  for (int i=0; i<nTags; i++) {
 
    // Fetch each tag and its properties.
    // Compute its display size based on its tag cloud "weight";
    // Then reshape the display size non-linearly, for display purposes.
    Tag aTag = (Tag) tags.get(i);
    String tName = aTag.getName();
    float tWeight = (float) aTag.getWeight();
    float wordSize =  maxWordDisplaySize * ( pow (tWeight/maxWordDisplaySize, 0.6));
 
    //we calculate the length of the text up here so the buttons can be made with it.
    float xPos0 = xPos;
    textSize(wordSize);
    float xPos1 = xPos + textWidth (tName) + 2.0;
    textSize(wordSize/2);
    float xPos2 = xPos1 + textWidth (str((float)aTag.getScore())) + 2.0;
 
    //make a transparent button for each word. This can be used to tell if we are hovering over a word, and what word.
    if(changeTextButt)//We only make new buttons if we've done a new search (saves time, and they stick around).
    {
      textButt[i] = controlP5.addButton("b-"+str(i),(float)i,(int)xPos0,(int)(yPos-wordSize),(int)(xPos2-xPos0),(int)wordSize);
      textButt[i].setColorBackground(transparent);
      textButt[i].setColorForeground(transparent);
      textButt[i].setColorActive(transparent);
      textButt[i].setLabel("");
    }
    else//if we aren't making new buttons, we're checking to see if the mouse is inside the button for the current word.
    {
 
      if(textButt[i].isInside())
      {
        if(listExists == -1)//If there is no popup list on screen, we make one and fill it
        {
          hoverList = controlP5.addListBox(tName,(int)xPos0-40,(int)(yPos-wordSize),(int)(xPos2-xPos0+20),60);
          hoverList.setItemHeight(12);
          hoverList.setBarHeight(12);
          hoverList.setColorBackground(buttbgcolor);
          hoverList.setColorForeground(buttfgcolor);
          hoverList.setColorActive(defaultactive);
          fillHoverList(tName, xPos2-xPos0+25.0);
          listExists = i;//This is which button/word the list is on.
        }
        /*else
         {
         //inside a button and list is here. could add keyboard scroll behavior.
         }*/
      }
      else if(listExists == i)//outside this button, and list is here. delete list.
      {
        listExists = -1;
        hoverList.hide();
        hoverList.remove();
      }
    }
 
 
    // Draw the word
    textSize(wordSize);
    fill ((i%2)*255,0,0); // alternate red and black words.
    text (tName, xPos,yPos);
 
    //Advance the writing position.
    xPos += textWidth (tName) + 2.0;
 
 
    //Draw the frequency
    textSize(wordSize/2);
    text (str((int)aTag.getScore()),xPos,yPos);
 
    // Advance the writing position
    xPos += textWidth (str((float)aTag.getScore())) + 2.0;
    if (xPos > (width - (xMargin+10))) {
      xPos  = xMargin;
      yPos += ySpacing;
    }
  }
  if(changeTextButt)//If we made new buttons, we don't need to make new buttons next draw().
  {
    changeTextButt = false;
  }
}
 
//Fills the popup list with games.
void fillHoverList(String word, float tWidth)
{
  int hCount = 0;
  for(int i = 0; i < currentSearch.size(); i++)
  {
    boolean nameCheck = false;
    String[] nameSplit = split((String)currentSearch.get(i)," ");
    for(int j = 0; j < nameSplit.length; j++)
    {
      String aWord = nameSplit[j];
      while (aWord.endsWith(".") || aWord.endsWith(",") || aWord.endsWith("!") || aWord.endsWith("?")|| aWord.endsWith(":")) {
        aWord = aWord.substring(0, aWord.length()-1);
      }
      aWord = aWord.toLowerCase();
      if(aWord.equals(word))
      {
        nameCheck = true;
        break;
      }
    }
    if(nameCheck)
    {
      String addName = (String)currentSearch.get(i);
      textSize(10);
      if(addName.length() > (int)(tWidth/7.35))
      {
        addName = addName.substring(0,(int)(tWidth/7.35-1))+"\u2026";
      }
      hoverList.addItem(addName,i);
      hCount++;
    }
  }
  hoverList.captionLabel().set(word+" - "+hCount);
}
 
//this searches the data for games that contain any of the parameter ratings, and all of the parameter descriptors.
void search(int theValue) {
  listCnt = 0;
  currentSearch.clear();
  cloud = new Cloud();
  cloud.setMaxWeight(maxWordDisplaySize); // max font size
  cloud.setMaxTagsToDisplay (130);
  String[] searchedGames = new String[combLength];
  for(int i = 0; i < combLength; i++)
  {
    String[] ratingCheck = {
      "none"
    };
    for(int r = 0; r < rateSearch.size(); r++)
    {
      ratingCheck = match(rating[i],(String)rateSearch.get(r));
      if(ratingCheck != null)
      {
        break;
      }
    }
    String[] descripCheck = {
      "none"
    };
    for(int d = 0; d < descripSearch.size(); d++)
    {
      descripCheck = match(descriptors[i],(String)descripSearch.get(d));
      if(descripCheck == null)
      {
        break;
      }
    }
    if(descripCheck != null && ratingCheck != null)
    {
      searchedGames[listCnt] = names[i];
      currentSearch.add(names[i]);
      String nameWords[] = split(searchedGames[listCnt], " ");
      for(int z = 0; z < nameWords.length;z++)
      {
        String aWord = nameWords[z];
        while (aWord.endsWith(".") || aWord.endsWith(",") || aWord.endsWith("!") || aWord.endsWith("?")|| aWord.endsWith(":") || aWord.endsWith(")")) {
          aWord = aWord.substring(0, aWord.length()-1);
        }
        while (aWord.startsWith(".") || aWord.startsWith(",") || aWord.startsWith("!") || aWord.startsWith("?")|| aWord.startsWith(":") || aWord.startsWith("(")) {
       aWord = aWord.substring(1, aWord.length());
       }
        aWord = aWord.toLowerCase();
        if(aWord.length() > 2 &&!(aWord.equals("of")) && !(aWord.equals("and")) && !(aWord.equals("the")) && !(aWord.equals("game")) && !(aWord.equals("games"))) {
          cloud.addTag(new Tag(aWord));
        }
      }
      listCnt++;
    }
  }
  changeTextButt = true;//time to make new buttons.
  for(int i = 0; i < textButt.length; i++)//delete old buttons.
  {
    textButt[i].hide();
    textButt[i].remove();
  }
}

Eric Brockmeyer – Project 2 (finished work)

by eric.brockmeyer @ 12:31 am

I have always found the stairs in Pittsburgh to be unexpectedly beautiful and exciting to discover. They are tucked between houses, up steep hills, and along busy streets all over the city. Pittsburgh public stairs are iconic of a city built on hills and should be maintained and respected as such. Unfortunately, they have fallen into a state of dilapidation and disrepair. It could be that fewer people take the stairs since their construction due to increase in ownership in cars, or it could be a National trend toward underfunded and under maintained infrastructure. In a report in 2009, the American Society for Civil Engineers ranked the US as a D (on an A-F scale).

The purpose of this project was to find a source of data that was easily accessible and convert it into some form of useful data. I used a website which contains a collection of photos of Pittsburgh public stairs and used these images to drive a Mechanical Turk survey regarding the state of disrepair of these stairs. The data analysis was somewhat subjective and based on inconsistent images of a sampling of stairs, however it does provide some general feedback on what the most prevalent problems are.

I used 45 images of 45 different sets of stairs. Each of these had 5 surveys performed to help discover anomalous answers and to provide a larger overall data set. The survey consisted of 8 questions 2 of which were controls to cull bad responses (a learned necessity when using a service such as Mechanical Turk. They are as follows:

1. Is the hand rail rusted?
Yes
No

2. Is the hand rail bent or broken?
Yes
No

3. Are plants growing on or over the path?
Yes
No

4. Is the concrete chipped or broken?
Yes
No

5. Is there a pink bunny in the image? (control)
Yes
No

6. Is there exposed rebar?
Yes
No

7. Is this an image of stairs? (control)
Yes
No

8. Would you feel safe using these stairs?
Yes
No

To analyze and visualize the data I made a couple of processing sketches which performed different functions. I created a sketch to download and save all the images used in the survey into a folder of thumbnails and full size images. Next, I wrote a sketch to extract the results from my Mechanical Turk survey. The results came as a .csv file and was easily traversed and accessed using the XlsReader library. Finally I created two different visualizations to get some idea on what this data meant.

One visualization (at the top of this page) describes the overall responses to the questionnaire across all 45 images. Each circle represents the number of positive responses from each question for each question in each image. The spacing of the circles is randomly generated based on the total number of affirmative responses for each question.

The other visualization (below) contains an array of all 45 images which a user can select and see the data related to each individual photo. It also provides a bar graph on top of each thumbnail which is the sum of all affirmative responses for that image. Thus the larger the bar the more decrepit those stairs should appear.

The results of this project were interesting but not too surprising. I was very interested in the subject matter but I think the work flow from web images to mechanical turk to processing was fun to navigate. This project has uncovered some of the challenges in this work flow and there is definitely room to improve the interaction between these pieces of software.

Otherwise, I think the data provides some feedback on the number of stairs which require maintenance or larger repairs. I’m considering another iteration on this concept which encourages individuals to participate in a game where they are actually surveying public works like the Pittsburgh public stairs.

eric_brockmeyer_project_02

YATV – Final Post

by dpieri @ 9:35 pm 30 January 2011

YATV – Yet Another Twitter Visualizer – is a Twitter visualizer inspired by the Baby Name Voyager. It plots the popularity of your search term in Twitter posts over the last few hours.

http://yatv.heroku.com/

Showing recent Tweets about the Steelers

Background
Though I do not use Twitter myself there are many things about it that I find fascinating. I have had Twistori as my screensaver for a few years now. Some of my favorite other Twitter related things are: Who’s Pooping On Twitter, Newlyweds On The Job, and Queen’s English 50¢.

Inspiration
The interface for YATV is inspired by the Baby Name Voyager I mentioned above. There are many many Twitter visualizations out there that are pretty involved, such as Just Landed however I thought there was a need for something dead-simple, like YATV.
I also chose to do a Twitter visualization because I wanted to use this project as an chance to learn how to interface with the Twitter API, learn how to draw in a browser using pure Javascript, and to focus on making a simple interaction.

Execution
I decided to build the project using Ruby on Rails. I have been using it for about 8 months now, and I haven’t looked back. Honestly, the idea of going back to Processing and having to deal with things like static typing was a bit frightening.
I found a good resource for learning drawing in Javascript here.
The site is hosted on Heroku.

I’ve put excerpts of the code in a separate blog post. Here

Problems
Once I started reading into the Twitter API I realized that my concept wouldn’t be as useful as I had hoped. Using the Search API one can only get up to 15 pages of results with 100 Tweets per post. With this restriction, searching for popular terms like Justin Bieber only allows you to see Tweets going back less than 10 minutes.
8 Minutes of Tweets about Justin Bieber

Final Product
I wanted to keep the interface as simple as possible, so there is really only one interaction. There is a search bar, and each time you search a term the new graph stacks at the top of the page on top of the other ones.

jparsons – Project 2 – Project Gutenberg

by Jordan Parsons @ 9:20 pm

So for this project I chose to look at a rdf dump of the project Gutenberg catalog. This is their online listing of almost 30,000 books that are public domain. Their website is very old and clunky to use. So I used processing and a MySQL database to go over their site and try to visualize the data from the catalog looking for some statistics and comparisons where I could.

The Data

I processed the .RDF Project Gutenberg catalog into a MySQL database.

Going from:

<pgterms:etext rdf:ID="etext34067">
<dc:publisher>&pg;</dc:publisher>
<dc:title rdf:parseType="Literal">Catholic Churchmen in Science</dc:title>
<dc:creator rdf:parseType="Literal">Walsh, James J.</dc:creator>
<pgterms:friendlytitle rdf:parseType="Literal">Catholic Churchmen in Science by James J. Walsh</pgterms:friendlytitle>
<dc:language><dcterms:ISO639-2><rdf:value>en</rdf:value></dcterms:ISO639-2></dc:language>
<dc:created><dcterms:W3CDTF><rdf:value>2010-10-13</rdf:value></dcterms:W3CDTF></dc:created>
<dc:rights rdf:resource="&lic;" />
</pgterms:etext>

to :

<book_id id="etext34067">
</book_id>
<book_title id="title">Catholic Churchmen in Science</book_title>
<book_auth id="auth">Walsh, James J.</book_auth>
<book_etitle id="etitle">Catholic Churchmen in Science by James J. Walsh</book_etitle>
<book_lang id="lang">en</book_lang>
<book_created id="create">2010-10-13</book_created>

Then processed through processing’s XML functions into a MySQL database.

The table has 30,248 entries and its about 6 mb in size. This should give me a huge dataset to start to play with and analyze.

MySQL Format:
id 1
book_id etext34714
book_title Catholic Churchmen in Science
book_auth Walsh, James J.
book_etitle Catholic Churchmen in Science by James J. Walsh
book_lang en
book_created 10/13/2010

The Project
The data is visualized as if it is a simple book itself. Once the book is opened the table of contents appears, listing the different methods of browsing the catalog. Each method is listed with the number of different sub categories it contains within itself. This top layer shows a bar graph across the bottom showing the relative scale of the different categories. This same graph changes across the project to show the amount of books each sub category (such as author, or language) has in it. Once a category is picked the user can page through the entries in that category. Upon clicking an author the user is given one of their books to read, the url of which is provided through google, because since releasing the data, they have changed their book id, system so it is impossible to link directly to the book.

My Critique
This project is by no means as successful as I had desired. I think it was a good experience for me to develop this kind of project as I have not programed that many things with user interfaces before. In addition I learned a lot about how to work with strings and data, something I had not had that much exposure to. I’m somewhat happy with the result, but the level of polish I had hoped for and many of the features I wanted are just not there. I think I would take an other iteration for me to get this project to a state where I am happy with it.

http://prezi.com/pdbjwudnrsef/project-2-project-gutenberg-info-viz/

Code:

 
import de.bezier.data.sql.*;
 
book book;
graph graph;
//Book text_book;
 
PImage frame;
PFont base_text,big_text;
MySQL mysql;
int total_entries, book_right_tip,book_right_side,book_bottom_tip;
int margin, page_l,page_r;
int high;
ArrayList g_n, g_t;
String location;
boolean book_open, graph_on;
 
void setup() {
  //Setup
  background(255);
  smooth();
  size(1000,600);
 
  //Vars
  margin = 10;
  ArrayList g_t = new ArrayList();
  ArrayList g_n = new ArrayList();
  location = "Click To Begin";
  book_open = false;
  book = new book(270,400);
  graph_on = false;
  high = 0;
 
  //Text Stuff
  base_text = loadFont("Swiss721BT-Roman-48.vlw");
  big_text = loadFont("Swiss721BT-Bold-48.vlw");
 
  //MySQL Stuff
  String user     = "root";
  String pass     = "989777989";
  String database = "rdf_dump";
  mysql = new MySQL( this, "localhost", database, user, pass );
  mysql.connect();
  mysql.query( "SELECT COUNT(DISTINCT id) FROM books" );
  mysql.next();
  total_entries = mysql.getInt(1);
 
  //Misc
  fill(100);
  textFont(big_text,24);
  text("Project Gutenberg",5,25);
  textFont(base_text,14);
  text("A book of "+total_entries+" books.",5,40);
  book.draw_book(false,0,0);
}
 
void update() {
  //Run Queries
}
 
void draw() {
  //Math
  update();
  //Make Pretty Things
 
  //Header
  background(255);
  //Graph BG
  if(graph_on==true) {
    graph.render(high);
  }
  //Rest
  book.draw_book(book_open,0,0);
  fill(100);
  textFont(big_text,24);
  text("Project Gutenberg",5,25);
  textFont(base_text,14);
  text("A book of "+total_entries+" books.",5,40);
  //Book Base
 
  book.fill_book();
  disp_info();
  if(book.fill_book = true && book.book_layer >1) {
    book.book_link_on = true;
  }
}
 
 
void disp_info() {
  //draw
 
  textFont(base_text,10);
  text(location, book.book_right_side, book.book_right_tip+15);
  if(book_open == false) {
    text("Instructions:", book.book_right_side, book.book_right_tip+30);
    text("Click the cover to start,", book.book_right_side, book.book_right_tip+45);
    text("then browse through the data using your mouse.", book.book_right_side, book.book_right_tip+60);
    text("Arrow keys move pages, space closes the book.", book.book_right_side, book.book_right_tip+75);
  }
}
 
 
void anim_open_book() {
  //animate the book opening
}
 
boolean overTitle() 
{
  if (mouseX >= 0 && mouseX <= 150 && 
    mouseY >= 0 && mouseY <= 50) {
    return true;
  } 
  else {
    return false;
  }
}
 
 
void mousePressed() {
  if(overTitle()==true) {
    setup();
  } 
  else {
 
    if(book_open == false) {
      book_open = true;
      book.book_layer = 1;
      location = "Index";
      page_l = 1;
      page_r = 10;
      book.fill_book = false;
    } 
    else {
 
      if(book.overAuth() == true&& book.book_layer == 1) {
        book.book_layer = 2;
      } 
      else
        if(book.overLang() == true&& book.book_layer == 1) {
          book.book_layer = 3;
        } 
        else
          if(book.overContrib() == true&& book.book_layer == 1) {
            book.book_layer = 4;
          } 
          else
            if(book.overDate() == true&& book.book_layer == 1) {
              book.book_layer = 5;
            }
      if(book.overIndex() == true && book.book_layer != 1 && book.book_layer != 0) {
        book.book_layer = 1;
      } 
      else
        if(book.overPage_1()==true) {
          //clicked right page 
          book.page_l();
          book.c2_back = true;
          book.c3_back = true;
          book.c4_back = true;
          book.c5_back = true;
          book.c2 = false;
          book.c3 = false;
          book.c4 = false;
          book.c5 = false;
          book.fill_book = false;
        }
        else if(book.overPage_2()==true) {
          book.page_r(); 
          book.c2 = false;
          book.c3 = false;
          book.c4 = false;
          book.c5 = false;
          println("bla");
 
          if(book.book_link_on == true) {
            String[] test = split(book.book_link_id, "etext");
            String test2 = book.book_link_title;
            test2 = test2.replace(" ","+");
            link("http://www.google.com/search?q="+test2+"+site:http://www.gutenberg.org/");
            println("http://www.gutenberg.org/ebooks/search.html/?format=html&default_prefix=titles&sort_order=downloads&query="+test2+"");
            book.book_link_on = false;
          }
        } 
        else {
        }
      if(book.overItem()>-1) {
      }
    }
  }
}
 
 
 
 
void keyPressed() {
  if(key == ' ') {
    setup();
  }
 
  if(keyCode == LEFT) {
    book.page_l();
    switch(book.book_layer) {
 
    case 0:
      break;
 
    case 1:
 
 
      break;
 
    case 2:
      book.c2_back = true;
      book.c2 = false;
      break;
 
    case 3:
      break;
 
    case 4:
      break;
    }
  }
 
  if(keyCode == RIGHT) {
    book.page_r();
    switch(book.book_layer) {
 
    case 0:
      break;
 
    case 1:
 
 
      break;
 
    case 2:
      book.c2 = false;
      break;
 
    case 3:
      break;
 
    case 4:
      break;
    }
  }
  if(key == '2') {
    book.book_layer = 2;
  }
}
//Layers-------------
//0 closed
//1 index
//2 authors
//4 titles
//3 lang
//5 date
//-------------------
 
class book {
  int book_right_side, book_right_tip, book_bottom_tip; 
  int book_x, book_y, margin, offset, text_width, book_layer;
  ArrayList book_stats, book_auths, book_auths_n, book_lang, book_lang_n, book_contrib, book_contrib_n, book_date, book_date_n;
  int max_stuff, start, end,item;
  String out ,book_link_id, book_link_title;
 
  boolean fill_book,book_link_on;
 
 
  boolean c1,c2,c3,c4,c5;
  boolean c2_back,c3_back,c4_back,c5_back;
 
  book(int bx, int by) {
    margin = 10;
    offset = 100;
    book_x = bx;
    book_y = by;
    book_right_side = book_x+margin;
    book_right_tip = offset;
    book_bottom_tip = offset+book_y;
    book_layer = 0;
 
    c1=c2=c3=c4=c5=false;
    book_stats = new ArrayList();
    book_auths = new ArrayList();
    book_auths_n = new ArrayList();
    book_lang = new ArrayList();
    book_lang_n = new ArrayList();
    book_contrib = new ArrayList();
    book_contrib_n = new ArrayList();
    book_date = new ArrayList();
    book_date_n = new ArrayList();
    max_stuff = 30;
    start = 0;
    c2_back = false;
    c3_back = false;
    book_link_on = false;
    fill_book = false;
    end = max_stuff;
    out = "";
  }
 
  void draw_book(boolean book_open, int book_pages,int book_read_pages) {
 
    if(!book_open) {
      stroke(200);
      fill(255);
      book_right_side = book_x+margin+offset;
      book_right_tip = offset;
      book_bottom_tip = offset+book_y;
      rect(offset,offset,book_x,book_y);
      line(offset+10,offset,offset+10,book_bottom_tip);
    }
    else {
      stroke(200);
      fill(255);
      book_right_side = book_x*2+margin+offset;
      book_right_tip = offset;
      book_bottom_tip = offset+book_y;
      rect(offset,offset,book_x,book_y);
      rect(offset+book_x,offset,book_x,book_y);
      draw_pages(page_l,page_r);
    }
  }
 
  void fill_book() {
    //text on pages
    if(book_open == false) {
      //cover!
      textFont(big_text,20);
      textAlign(CENTER);
      text("Project Gutenberg",offset,offset+book_y/2-20,book_x,book_y);
 
      textFont(base_text,12);
      text("Catalog",offset,offset+book_y/2+12,book_x,book_y);
      textAlign(LEFT);
    } 
    else {
      //Data
      get_data("");
      render_data();
 
      //Labels
 
      switch(book_layer) {
 
      case 0:
        break;
 
      case 1:
        textFont(big_text,14);
        color(150);
        textAlign(RIGHT);
        text("Table of Contents",offset+book_x-margin,offset+14+margin);
        textAlign(LEFT);
 
 
        break;
 
      case 2:
        textFont(big_text,14);
        color(150);
        textAlign(RIGHT);
        text("Authors",offset+book_x-margin,offset+14+margin);
        textAlign(LEFT);
        for(int i = 0; i<book_auths.size(); i++) {
          textFont(base_text,12);
          color(150);
          textAlign(RIGHT);
          text((String) book_auths.get(i),width-margin,offset+(13*i));
          textAlign(LEFT);
        }
        break;
 
      case 3:
        textFont(big_text,14);
        color(150);
        textAlign(RIGHT);
        text("Languages",offset+book_x-margin,offset+14+margin);
        textAlign(LEFT);
        for(int i = 0; i<book_lang.size(); i++) {
          textFont(base_text,12);
          color(150);
          textAlign(RIGHT);
          text((String) book_lang.get(i),width-margin,offset+(13*i));
          textAlign(LEFT);
        }
        break;
 
      case 4:
        textFont(big_text,14);
        color(150);
        textAlign(RIGHT);
        text("Contributiors",offset+book_x-margin,offset+14+margin);
        textAlign(LEFT);
        for(int i = 0; i<book_contrib.size(); i++) {
          textFont(base_text,12);
          color(150);
          textAlign(RIGHT);
          text((String) book_contrib.get(i),width-margin,offset+(13*i));
          textAlign(LEFT);
        }
        break;
 
      case 5:
        textFont(big_text,14);
        color(150);
        textAlign(RIGHT);
        text("Dates Added",offset+book_x-margin,offset+14+margin);
        textAlign(LEFT);
        for(int i = 0; i<book_date.size(); i++) {
          textFont(base_text,12);
          color(150);
          textAlign(RIGHT);
          text((String) book_date.get(i),width-margin,offset+(13*i));
          textAlign(LEFT);
        }
        break;
      }
      if(fill_book == true) {
        if(out.equals("")) {
        } 
        else {
          textFont(base_text,12);
          text(out,offset+book_x+margin,offset+14+margin,offset+book_x*2-margin,offset+book_y);
          textAlign(LEFT);
        }
      }
    }
  }
 
 
  void textFill(int item) {
    //link("http://www.processing.org", "_new");
 
    switch(book_layer) {
 
    case 0:
      fill_book = false;
      break;
 
 
    case 1:
      fill_book = false;
      break;
 
    case 2:
 
      mysql.query( "SELECT * FROM `books` WHERE `book_auth` = '"+book_auths.get(item)+"'" );
      mysql.next();
 
      out = mysql.getString("book_title");
      book_link_id = mysql.getString("book_id");
      book_link_title = (String) mysql.getObject("book_etitle");
      println(book_link_title);
      fill_book = true;
 
      break;
 
    case 3:
      out = "";
 
      textFont(base_text,12);
      text(out,offset+book_x+margin,offset+14+margin,offset+book_x*2,offset+book_y);
      textAlign(LEFT);
      break;
 
    case 4:
      out = "";
 
      textFont(base_text,12);
      text(out,offset+book_x+margin,offset+14+margin,offset+book_x*2,offset+book_y);
      textAlign(LEFT);
      break;
 
    case 5:
      String out = "";
 
      textFont(base_text,12);
      text(out,offset+book_x+margin,offset+14+margin,offset+book_x*2,offset+book_y);
      textAlign(LEFT);
      break;
    }
  }
 
 
  void get_data(String misc) {
 
    switch(book_layer) {
 
    case 0:
      break;
 
    case 1:
      //Index
      if(c1 == false) {
        mysql.query( "SELECT COUNT(DISTINCT book_auth) FROM books" );
        mysql.next();
        book_stats.add(mysql.getInt(1));
 
        mysql.query( "SELECT COUNT(DISTINCT book_lang) FROM books" );
        mysql.next();
        book_stats.add(mysql.getInt(1));
 
        mysql.query( "SELECT COUNT(DISTINCT book_contrib) FROM books" );
        mysql.next();
        book_stats.add(mysql.getInt(1));
 
 
        mysql.query( "SELECT COUNT(DISTINCT book_created) FROM books" );
        mysql.next();
        book_stats.add(mysql.getInt(1));
 
        graph = new graph(book_stats);
        graph_on = true;
 
        c1 = true;
      } 
 
      break;
 
    case 2:
      //auths
      if(c2 == false) {
        //right page clicked
        if(c2_back == false) {
          println( "+SELECT DISTINCT book_auth FROM books LIMIT "+start+" , "+end );
          mysql.query( "SELECT DISTINCT book_auth FROM books LIMIT "+start+" , "+end );
 
          //mysql.next();
          while(mysql.next()) {
            if(book_auths.size()>max_stuff) {
              book_auths.remove(0);
              book_auths.add(mysql.getString("book_auth"));
            }
            else {
              book_auths.add(mysql.getString("book_auth"));
            }
          }
          start = end;
          end = end+max_stuff;
        } 
        else {
          //left page clicked
          if((start-max_stuff)<0) {
            println("-SELECT DISTINCT book_auth FROM books LIMIT "+(0)+" , "+(max_stuff));
            mysql.query( "SELECT DISTINCT book_auth FROM books LIMIT "+(0)+" , "+(max_stuff) );
            start=30;
            end=60;
          }
          else {
            println( "-aSELECT DISTINCT book_auth FROM books LIMIT "+(start-max_stuff)+" , "+(end-max_stuff) );
            mysql.query( "SELECT DISTINCT book_auth FROM books LIMIT "+(start-max_stuff)+" , "+(end-max_stuff) );
          }
          //mysql.next();
          while(mysql.next()) {
            if(book_auths.size()>max_stuff) {
              book_auths.remove(0);
              book_auths.add(mysql.getString("book_auth"));
            }
            else {
              book_auths.add(mysql.getString("book_auth"));
            }
          }
          end = start;
          start = start-max_stuff;
          c2_back = false;
        }
        //draw it
        for(int i = 0; i < book_auths.size(); i++) {
          mysql.query( "SELECT COUNT(`book_auth`) FROM `books` WHERE `book_auth` = '"+book_auths.get(i)+"'" );
          mysql.next();
          if(book_auths.size()>max_stuff) {
            book_auths_n.remove(0);
            book_auths_n.add((int)mysql.getInt(1));
          } 
          else {
            book_auths_n.add((int)mysql.getInt(1));
          }
        }
 
        graph.is_dead = true;
        graph = new graph(book_auths_n);
        graph_on = true;
        c2 = true;
      }
      break;
      ////////////////////////////////////////////////////////////////////////////////////
    case 3:
 
      //lang Echo
      if(c3 == false) {
        //right page clicked
        if(c3_back == false) {
          println( "+SELECT DISTINCT book_lang FROM books LIMIT "+start+" , "+end );
          mysql.query( "SELECT DISTINCT book_lang FROM books LIMIT "+start+" , "+end );
 
          //mysql.next();
          while(mysql.next()) {
            if(book_lang.size()>max_stuff) {
              book_lang.remove(0);
              book_lang.add(mysql.getString("book_lang"));
            }
            else {
              book_lang.add(mysql.getString("book_lang"));
            }
          }
          start = end;
          end = end+max_stuff;
        } 
        else {
          //left page clicked
          if((start-max_stuff)<0) {
            println("-SELECT DISTINCT book_lang FROM books LIMIT "+(0)+" , "+(max_stuff));
            mysql.query( "SELECT DISTINCT book_lang FROM books LIMIT "+(0)+" , "+(max_stuff) );
            start=30;
            end=60;
          }
          else {
            println( "-aSELECT DISTINCT book_lang FROM books LIMIT "+(start-max_stuff)+" , "+(end-max_stuff) );
            mysql.query( "SELECT DISTINCT book_lang FROM books LIMIT "+(start-max_stuff)+" , "+(end-max_stuff) );
          }
          //mysql.next();
          while(mysql.next()) {
            if(book_lang.size()>max_stuff) {
              book_lang.remove(0);
              book_lang.add(mysql.getString("book_lang"));
            }
            else {
              book_lang.add(mysql.getString("book_lang"));
            }
          }
          end = start;
          start = start-max_stuff;
          c3_back = false;
        }
        //draw it
        for(int i = 0; i < book_lang.size(); i++) {
          mysql.query( "SELECT COUNT(`book_lang`) FROM `books` WHERE `book_lang` = '"+book_lang.get(i)+"'" );
          mysql.next();
          if(book_lang.size()>max_stuff) {
            book_lang_n.remove(0);
            book_lang_n.add((int)mysql.getInt(1));
          } 
          else {
            book_lang_n.add((int)mysql.getInt(1));
          }
        }
 
        graph.is_dead = true;
        graph = new graph(book_lang_n);
        graph_on = true;
        c3 = true;
      }
      break;
      /////////////////////////////////////////////////////////////////////////////////
    case 4:
      //lang Date
      if(c4 == false) {
        //right page clicked
        if(c4_back == false) {
          println( "+SELECT DISTINCT book_contrib FROM books LIMIT "+start+" , "+end );
          mysql.query( "SELECT DISTINCT book_contrib FROM books LIMIT "+start+" , "+end );
 
          //mysql.next();
          while(mysql.next()) {
            if(book_contrib.size()>max_stuff) {
              book_contrib.remove(0);
              book_contrib.add(mysql.getString("book_contrib"));
            }
            else {
              book_contrib.add(mysql.getString("book_contrib"));
            }
          }
          start = end;
          end = end+max_stuff;
        } 
        else {
          //left page clicked
          if((start-max_stuff)<0) {
            println("-SELECT DISTINCT book_contrib FROM books LIMIT "+(0)+" , "+(max_stuff));
            mysql.query( "SELECT DISTINCT book_contrib FROM books LIMIT "+(0)+" , "+(max_stuff) );
            start=30;
            end=60;
          }
          else {
            println( "-aSELECT DISTINCT book_contrib FROM books LIMIT "+(start-max_stuff)+" , "+(end-max_stuff) );
            mysql.query( "SELECT DISTINCT book_contrib FROM books LIMIT "+(start-max_stuff)+" , "+(end-max_stuff) );
          }
          //mysql.next();
          while(mysql.next()) {
            if(book_contrib.size()>max_stuff) {
              book_contrib.remove(0);
              book_contrib.add(mysql.getString("book_contrib"));
            }
            else {
              book_lang.add(mysql.getString("book_contrib"));
            }
          }
          end = start;
          start = start-max_stuff;
          c4_back = false;
        }
        //draw it
        for(int i = 0; i < book_contrib.size(); i++) {
          mysql.query( "SELECT COUNT(`book_contrib`) FROM `books` WHERE `book_contrib` = '"+book_contrib.get(i)+"'" );
          mysql.next();
          if(book_contrib_n.size()>max_stuff) {
            book_contrib_n.remove(0);
            book_contrib_n.add((int)mysql.getInt(1));
          } 
          else {
            book_contrib_n.add((int)mysql.getInt(1));
          }
        }
 
        graph.is_dead = true;
        graph = new graph(book_contrib_n);
        graph_on = true;
        c4 = true;
      }
 
      break;
 
      /////////////////////////////////////////////////////////////////////////////////
    case 5:
      //date Data
      if(c5 == false) {
        //right page clicked
        if(c5_back == false) {
          println( "+SELECT DISTINCT book_created FROM books LIMIT "+start+" , "+end );
          mysql.query( "SELECT DISTINCT book_created FROM books LIMIT "+start+" , "+end );
 
          //mysql.next();
          while(mysql.next()) {
            if(book_date.size()>max_stuff) {
              book_date.remove(0);
              book_date.add(mysql.getString("book_created"));
            }
            else {
              book_date.add(mysql.getString("book_created"));
            }
          }
          start = end;
          end = end+max_stuff;
        } 
        else {
          //left page clicked
          if((start-max_stuff)<0) {
            println("-SELECT DISTINCT book_created FROM books LIMIT "+(0)+" , "+(max_stuff));
            mysql.query( "SELECT DISTINCT book_created FROM books LIMIT "+(0)+" , "+(max_stuff) );
            start=30;
            end=60;
          }
          else {
            println( "-aSELECT DISTINCT book_created FROM books LIMIT "+(start-max_stuff)+" , "+(end-max_stuff) );
            mysql.query( "SELECT DISTINCT book_created FROM books LIMIT "+(start-max_stuff)+" , "+(end-max_stuff) );
          }
          //mysql.next();
          while(mysql.next()) {
            if(book_date.size()>max_stuff) {
              book_date.remove(0);
              book_date.add(mysql.getString("book_created"));
            }
            else {
              book_lang.add(mysql.getString("book_created"));
            }
          }
          end = start;
          start = start-max_stuff;
          c5_back = false;
        }
        //draw it
        for(int i = 0; i < book_date.size(); i++) {
          mysql.query( "SELECT COUNT(`book_created`) FROM `books` WHERE `book_created` = '"+book_date.get(i)+"'" );
          mysql.next();
          if(book_date.size()>max_stuff) {
            book_date_n.remove(0);
            book_date_n.add((int)mysql.getInt(1));
          } 
          else {
            book_date_n.add((int)mysql.getInt(1));
          }
        }
 
        graph.is_dead = true;
        graph = new graph(book_date_n);
        graph_on = true;
        c5 = true;
      }
 
      break;
    }
  }
 
  void render_data() {
 
    switch(book_layer) {
 
    case 0:
      break;
 
    case 1:
      textFont(base_text,14);
      text("Authors: "+book_stats.get(0),offset+book_x+margin,offset+14+margin);
      text("Languages: "+book_stats.get(1),offset+book_x+margin,offset+14+margin+20);
      text("Contributiors: "+book_stats.get(2),offset+book_x+margin,offset+14+margin+40);
      text("Dates Added: "+book_stats.get(3),offset+book_x+margin,offset+14+margin+60);
      textAlign(LEFT);
 
 
 
      break;
 
    case 2:
      break;
 
    case 3:
      break;
 
    case 4:
      break;
    }
  }
 
 
 
 
 
 
  void draw_pages(int p_left, int p_right) {
    //draw the pages on the ege of the book
 
    if(p_left <= 0) {
      p_left = 1;
    }
    if(p_left >= 10) {
      p_left = 10;
    }
    if(p_right <= 0) {
      p_right = 1;
    }
    if(p_right >= 10) {
      p_right = 10;
    }
 
    stroke(200);
    for(int i = p_left; i>0; i--) {
      int x = offset+4+i*2;
      line(x,offset,x,book_bottom_tip);
    }
 
    for(int i = p_right; i>0; i--) {
      int x = offset-4+book_x*2-i*2;
      line(x,offset,x,book_bottom_tip);
    }
  }
 
 
  void page_l() {
    page_l--;
    page_r++;
  }
 
  void page_r() {
    page_l++;
    page_r--;
  }
 
 
 
  int overItem() {
    if (mouseX >= offset+book_x*2 && mouseX <= width && 
      mouseY >= offset-13 && mouseY <= offset+book_y&&book_layer>1) {
 
      item =(int) (mouseY-offset)/11;
      println(mouseY+" "+item);
 
      if(item>0) {
 
        textFill(item);
      }
      return 0;
    } 
    else {
      item = -1;
      return -1;
    }
  }
 
  boolean overPage_1() 
  {
    if (mouseX >= offset && mouseX <= offset+book_x && 
      mouseY >= offset && mouseY <= offset+book_y) {
      return true;
    } 
    else {
      return false;
    }
  }
 
  boolean overPage_2() 
  {
    if (mouseX >= offset+book_x && mouseX <= offset+book_x*2 && 
      mouseY >= offset && mouseY <= offset+book_y) {
      return true;
    } 
    else {
      return false;
    }
  }
 
  boolean overAuth() {
 
    if (mouseX >= 375 && mouseX <= 430 && 
      mouseY >= 115 && mouseY <= 130) {
      return true;
    } 
    else {
      return false;
    }
  }
 
  boolean overLang() {
 
    if (mouseX >= 375 && mouseX <= 430 && 
      mouseY >= 140 && mouseY <= 155) {
      return true;
    } 
    else {
      return false;
    }
  }
 
  boolean overContrib() {
 
    if (mouseX >= 375 && mouseX <= 430 && 
      mouseY >= 160 && mouseY <= 175) {
      return true;
    } 
    else {
      return false;
    }
  }
 
  boolean overDate() {
 
    if (mouseX >= 375 && mouseX <= 430 && 
      mouseY >= 180 && mouseY <= 195) {
      return true;
    } 
    else {
      return false;
    }
  }
  boolean overIndex() {
 
    if (mouseX >= 240 && mouseX <= 360 && 
      mouseY >= 100 && mouseY <= 130) {
      return true;
    } 
    else {
      return false;
    }
  }
 
 
  void anim_open_book() {
    //animate the book opening
  }
}
class graph {
 
  int g_length, j,k,max_index;
  float x,y,vel;
  boolean is_new, is_dead, dead;
  ArrayList g_v = new ArrayList();
 
 
 
  graph(ArrayList g_vals) {
 
    x = 0;
    vel = .25;
    j=k=0;
    g_v = g_vals;
    //g_t = g_text;
    is_new = true;
    is_dead = dead = false;
 
    g_length = g_v.size();
 
    Integer max_int = 0;
    for(int i = 0; i<g_length; i++) {
      if((Integer) g_v.get(i) > (Integer) max_int) {
        max_int = (Integer) g_v.get(i);
      }
      max_index = g_v.indexOf(max_int);
    }
  }
 
  void update() {
    if(dead == true) {
    } 
    else {
      //x = x - vel;
      if(x<300) {
        is_dead = true;
      }
    }
  }
 
  void render(int high) {
    if(dead == true) {
    } 
    else {
 
      for(int i = 0; i < g_length; i++) {
        int h = (int) map((Integer) g_v.get(i),0,(Integer) g_v.get(max_index),0,300);
        fill(50);//(Integer) g_v.get(i)
        noStroke();
 
        //make graph text mouse over
        if(mouse_over_rect((int)x+15*(i+1)-12, height-h, 15, height)) {
          fill(200);
          rect(x+15*(i+1)-12, height-h, 15, h);
 
          textFont(base_text,15);
          text((Integer)g_v.get(i),mouseX+20,mouseY);
        } 
        else {
 
          rect(x+15*(i+1)-12, height-h, 15, h);
        }
      }
      //new
      if(is_new==true) {
        j++;
        fill(255,255-j);
        rect(0,0,width,height);
      }
      if(j>255) {
        is_new = false;
      }
      //dead
      if(is_dead==true) {
        k++;
        noStroke();
        fill(255,k);
        rect(0,0,width,height);
      }
      if(k>255) {
        dead = true;
        is_dead = false;
        is_new = true;
      }
    }
  }
  boolean mouse_over_rect(int x, int y, int w, int h) {
    if (mouseX >= x && mouseX <= x+w && 
      mouseY >= y && mouseY <= y+h) {
      return true;
    } 
    else {
      return false;
    }
  }
}
Next Page »
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
(c) 2017 Interactive Art & Computational Design / Spring 2011 | powered by WordPress with Barecity