Amanda Watson — Data Scraping

For this assignment, I was interested in looking at how people’s desires compared to their necessities.  I wanted to scrape tweets off of Twitter and look at the usage of “I want to” versus “I have to”.  What did people want, and what did they think was holding them back?

I collected about 1500 unique posts for each phrase using Temboo.  Looking at the output, I thought it might also be interesting to look at the wants-to’s and have-to’s people were prescribing others.  I collected another set of posts containing the phrase “they want to” and “they have to”

My ideas for visualization don’t really involve visualizing the sheer amount of data — I think it would be kind of interesting to use these phrases to construct sentence for narrative.  Display a randomly selected set of posts of the form “I want to…”, “but I have to…”, and examine how desires and necessities relate.  It might also be interesting to switch around the word “want” and “have” in the phrases, and see if the sentences remain plausible.


“i want to be harry’s friend so i can try to make him smile his beautiful smile and laugh so much his eyes crinkle shut every single day”

“I want to do something different….think i may sign up for some kickboxing classes”

“I want to eat but I’m not even hungry”

“@nina__schubert I want to see it ASAP!!! I’m so excited aha”

“i want to devote my time to individually messaging all 85k+ who liked that post and informing them of their poor life choice”

“I want a follow from @RickyPDillon but it’s 1:20 am so I think I have to sleep..”

“ugh now I have to go to track practice”

“So I’m at my last class of the day but then I remembered that I have to go see my advisor…”

“@PrairieHedge as an ex nurse I have to say hope!”

“Those words of reassurance is pretty much all I have to calm myself atm -A-”


“Can’t wait to leave college so I can get away from fucking idiots I have to see every day”

“Lol at all the reading I have to do before tomorrow”

“for art i have to do active research so i would appreciate if you could RT this if any of these have happened to you http:\\/\\/\\/VnzGdDDoXV”

“I have to get approval on certain things because it’s already hard to decipher tone thru text & me being as forward as I am it can go wrong.”

“Errbody wants to be rich but I rarely hear ppl say they want to be wealthy..#BigDifference…”

“they want to be accepted for shit that they actually do.\\” – OG Maco”

“Everyone is tweeting about how they want to go home and I’m just chillin in my bed :-)”

“@LaryssaBee exactly. People know where to go if they want to read the article. At least TG posts his blog but also summarizes in tweets”

“@chriscantrellOF fr they want to make it more of a hitters game prob”

“@abreumegan lol yes they will you’ll guide them down the right path \\ud83d\\udc4f you better let them smoke if they want to though”

“First impressions are big with #Virgos. They will know whether or not they want to be around you after just one conversation.”

“It’s Detroit v.s Everybody till one of them get where they want to go then its Us v.s the asshole who is doing fine.”

“\\”@TotoroHearts: @amsoulking they want to feel the music so deeply that their eardrums could explode \\n\\nthat’s me alright\\”\\n\\n#THIS”

“\\”Motivation is the art of getting people to do what you want them to do because they want to do it.\\”\\n\\n#PSFforSenate”

“RT @MaralaScott: When someone has something they want to share with you it is not because you are entitled but because you are blessed. \\u2013Ma\\u2026”

“They want to close”

“RT @KrIsTiNeOtErSoN: See they want to buy my pride”

“@Rathgrith027_SL @TheFartRises @b00nes They want to speak at the world but not have the world speak back.”

“@DigiRanger1994 @TheChaosBlue — They HAVE to cancel it.\\nHAVE TO.\\nThere’s a TRAVEL BAN as of 9PM tonight.”

“Girls who think they have to compete for a guy are just dumb if he likes you he’ll show quit trying”

“When people decide they want to start the unfriended movie in real life http:\\/\\/\\/d9atJi4Lo9”

“They want to go to Disneyland Feb 19-22..-.”

“#Hijackraw WWE @TripleH They want to play with the fans well then lets play with Them”

“Is the word \\”dyke\\” disrespectful to lesbians orrrr no? Like do they want to be called that?”

“Why do ex’s give the guilty trip when they want to make something right?”

“\\u201c@aquuuarius: #Aquarius tend to push people away when they’re moody or upset. They only’ll talk about it if they want to.\\u201d \\ud83d\\ude02\\ud83d\\ude02 so me”

“@ihateprospects There is a better chance of that than there is of him becoming beloved by the fans. They want to boo him after last night”

“Just cuz someone is single that doesn’t mean they want to be accosted in their DMs lol \\”@TucoFbaby: If you single why are your DM’s broke?\\””

“RT @leYOWard: We can’t beg someone to stay when\\nthey want to leave and be with someone else.”

“RT @akhbarlarh: @OhJujAi brooo please struggle sikit for your future . You know you mum and dad . They want to see you succed”

This entry was posted in 12-datascraping on by .

About amwatson

Amanda is a low-level software developer and theatre artist interested in building new experiences from the ground up. Currently an undergrad studying Computer Science and Drama (directing) through CMU's BXA program, Amanda comes to CMU from a conservatory theatre background; over the past four years, however, she's cultivated a passion for kernel hacking, GPU programming, and high-performance systems design. Her work often involves interpreting large amounts of data, and doing it fast. However, she wants to try everything, especially things that will challenge her assumptions and shatter any illusions of comfort with the world at large.