casmlab / purpletag Goto Github PK
View Code? Open in Web Editor NEWCalculate political polarization scores for members of U.S. Congress based on their tweets
License: BSD 3-Clause "New" or "Revised" License
Calculate political polarization scores for members of U.S. Congress based on their tweets
License: BSD 3-Clause "New" or "Revised" License
File "/usr/local/lib/python2.7/dist-packages/purpletag/purpletag_collect.py", line 37, in track_users
for tweet in twutil.collect.track_user_ids(ids):
add try/except around this line
score
command to list polarization scores for each member of congress.How do our estimates compare with Barbera's ideology estimates? Here's the paper: Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data.
Here's the error I got:
Traceback (most recent call last):
File "/home/libbyh/anaconda3/envs/purpletag/bin/purpletag-collect", line 11, in <module>
load_entry_point('purpletag==0.1.4', 'console_scripts', 'purpletag-collect')()
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/purpletag-0.1.4-py2.7.egg/purpletag/purpletag_collect.py", line 84, in main
track_users(ids)
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/purpletag-0.1.4-py2.7.egg/purpletag/purpletag_collect.py", line 40, in track_users
for tweet in twutil.collect.track_user_ids(ids):
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/TwitterAPI/TwitterAPI.py", line 305, in __iter__
for item in self._iter_stream():
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/TwitterAPI/TwitterAPI.py", line 280, in _iter_stream
raise TwitterConnectionError('Twitter stream stalled')
TwitterAPI.TwitterError.TwitterConnectionError: Twitter stream stalled
Not sure how long into tracking this happened. -collect
option worked fine.
I'm looking for finer control over historical analysis. We talked about two different ways to alter the way the code currently works.
Right now, when you enter a command like
purpletag parse -t 60 -d 100
purpletag starts with today and works its way backwards 100 days, one at a time. For roughly 4.5GB of JSON data, this takes over an hour per day.
User would issue a command like
purpletag parse -t 60 -s 2017-05-01
and purple tag would parse tweets from the 60 days before May 1, 2017. This option essentially overrides the "start with today" default and replaces it with "start with day user specifies".
Change line that starts the days loop to start at the oldest day instead of the most recent day. This option reverses the order of the existing process, and then users could quit the process if they wanted only the old days' data.
visualize
or graph
that generates some interesting graphs based on a set of .scores
files.Tracking users on Twitter returns their tweets along with retweets and mentions. If we ignore retweets and mentions, we'd have much less data to deal with; however, it might be nice to have these for some other purpose.
hangs with no output.
score
that simulates X days of historical scores
--track option
running less than a day
WARNING:root:<type 'exceptions.ValueError'> Expecting ':' delimiter: line 1 column 4964 (char 4963)
Traceback (most recent call last):
File "/home/libbyh/anaconda3/envs/purpletag/bin/purpletag-collect", line 11, in <module>
load_entry_point('purpletag==0.1.4', 'console_scripts', 'purpletag-collect')()
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/purpletag-0.1.4-py2.7.egg/purpletag/purpletag_collect.py", line 84, in main
track_users(ids)
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/purpletag-0.1.4-py2.7.egg/purpletag/purpletag_collect.py", line 40, in track_users
for tweet in twutil.collect.track_user_ids(ids):
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/TwitterAPI/TwitterAPI.py", line 311, in __iter__
raise TwitterConnectionError(e)
TwitterAPI.TwitterError.TwitterConnectionError: Expecting ':' delimiter: line 1 column 4964 (char 4963)
Using --track option
Running for a couple days
WARNING:root:<class 'requests.packages.urllib3.exceptions.ProtocolError'> ('Connection broken: IncompleteRead(0 bytes read, 1 more expected)', IncompleteRead(0 bytes read, 1 more expected))
Traceback (most recent call last):
File "/home/libbyh/anaconda3/envs/purpletag/bin/purpletag-collect", line 11, in <module>
load_entry_point('purpletag==0.1.4', 'console_scripts', 'purpletag-collect')()
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/purpletag-0.1.4-py2.7.egg/purpletag/purpletag_collect.py", line 84, in main
track_users(ids)
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/purpletag-0.1.4-py2.7.egg/purpletag/purpletag_collect.py", line 40, in track_users
for tweet in twutil.collect.track_user_ids(ids):
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/TwitterAPI/TwitterAPI.py", line 305, in __iter__
for item in self._iter_stream():
File "/home/libbyh/anaconda3/envs/purpletag/lib/python2.7/site-packages/TwitterAPI/TwitterAPI.py", line 294, in _iter_stream
raise TwitterConnectionError(e)
TwitterAPI.TwitterError.TwitterConnectionError: ('Connection broken: IncompleteRead(0 bytes read, 1 more expected)', IncompleteRead(0 bytes read, 1 more expected))
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.