Comments (22)
Hi Riccardo,
The track and follow parameters require a comma-separated string. Like,
'track':'keyword1,keyword2,keyword3'
There may be other issues, but try that first and see if it helps.
Jonas
from twitterapi.
Hi Jonas,
you're right, I forgot to mention that the array is joined under csv before passing it to the api.request.
Actually the script is working fine but I can't add more than 1100 users.
from twitterapi.
I wonder if you have reached some buffer size limit. Can you try using 1101 users, but substitute some users names with shorter names so the entire string is no larger than the string with 1100 users?
from twitterapi.
the exact limit seems to be:
157 keywords and 1181 accounts, if I add something to the query (keyword or account, or even spaces) I get the error.
the last working parameter size is: 13.376 byte
I'm trying to troubleshoot the problem under a new fork, maybe is something missing in the session.request?
from twitterapi.
Twitter's documentation for statuses/filter says 400 keywords and 5000 accounts. That is why I think you are hitting some buffer limit. Something to test would be if you can get more than 157 keywords or 1181 accounts without going over 13376 bytes, and see if that works. That would confirm my suspicion. HTTP has no limit on the query string size, and I haven't found a limit published by Twitter. It's also possible that the requests library imposes a limit, but I don't see that either.
from twitterapi.
I try splitting some keywords (example from #protezionecivile to "#protezione" and "civile") and got 503 error after just two split. Using Tweepy I was able to get all the 4500 accounts, but that library didn't fit my needings, I'm also taking a look at the request library, maybe the problem is there
from twitterapi.
ok.... don't ask me why, but I solved using the following code:
opz['track'] = ",".join(self.searchterm)
opz['follow'] = ",".join(self.searchterm2)
query = {'track': opz['track'], 'follow': opz['follow']}
r = self.api.request('statuses/filter', [query])
instead of
opz['track'] = ",".join(self.searchterm)
opz['follow'] = ",".join(self.searchterm2)
r = self.api.request('statuses/filter', {'track': opz['track'], 'follow': opz['follow']})
from twitterapi.
Well, that is odd. The params are fed directly into the requests library method. That method takes a dictionary, not a list. I'll take a closer look...
from twitterapi.
I have the same problem as well - my old Perl library does the job on the same input but I get a 503 whenever I follow more than 1534 people (about 13261 bytes worth of follow parameters). Again, same oauth token work with another library so the 503 error is probably not what it advertises itself to be (twitter overloaded). It definitely has to do with the numbers of bytes of parameters. The "fix" suggested by rmorandi doesn't work for me
from twitterapi.
jpschmetz can u paste the api.request and a part of the sent parameters?
from twitterapi.
api.request('statuses/filter', {'follow': '1,2,3,4,5....'}) -- checked how the framework processes the request and what is sent to Session.requests is OK but it somehow make twitter unavailable on the other side
works fine until the string gets too large (but not too large for my access).
from twitterapi.
did you already try with:
opz['follow'] = "1,2,3,4,5..."
query = {'follow': opz['follow']}
api.request('statuses/filter', [query])
it seems to be exactly the same problem
from twitterapi.
I have confirmed the problem as well. Wrapping params inside a list does make the problem go away for me. Problem seems to be in the requests library, so I will continue looking there. But for now, if you change line 98 in TwitterAPI.py to
params=[params]
should make things work until I find a real fix.
from twitterapi.
I changed the line in TwitterAPI but it creates another problem for me
File "./testsample.py", line 27, in
r = api.request('statuses/filter', query)
File "/usr/local/lib/python2.7/dist-packages/TwitterAPI/TwitterAPI.py", line 105, in request
proxies=self.proxies)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 422, in request
prep = self.prepare_request(req)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 360, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 294, in prepare
self.prepare_url(url, params)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 380, in prepare_url
enc_params = self._encode_params(params)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 85, in _encode_params
for k, vs in to_key_val_list(data):
ValueError: need more than 1 value to unpack
from twitterapi.
Jean-Paul,
Are you using OAuth1 or OAuth2?
from twitterapi.
pretty sure OAuth1 - is it possible to tell from the keys or how do I know for sure?
from twitterapi.
Default is OAuth1. You would need to specify OAuth2. Sorry I have no quick fix. Still looking into it.
from twitterapi.
I committed version 2.2.3 with the fix. Now HTTP POST requests always put parameters in the body of the request instead of in the query string. It should be good now.
from twitterapi.
Hi, I'm sorry but the new version gives always the same 503 error.
I also discovered that my suggested fix didn't work (twitter accept the parameters but didn't filter the keys and the accounts you send)
if you give me an e-mail address I'll send you my full query
Thanks,
Riccardo
from twitterapi.
update: I'm using this fork for now and works fine
https://github.com/rmorandi/TwitterAPI/blob/master/TwitterAPI/TwitterAPI.py
query = {'track': 'a,b,c... 157 keys', 'follow': '1,2,3... 4500 IDs', 'filter_level':'none', 'stall_warnings':'true'}
r = self.api.request('statuses/filter', query)
the only thing I changed is on line 87:89 and added some debug... maybe could help ;)
from twitterapi.
Hi Riccardo,
Send your query to [email protected].
Thanks!
Jonas
from twitterapi.
I have not been able to produce errors with the current version (2.2.3). I am closing this issue, but if anyone is still seeing errors please post. Thanks!
from twitterapi.
Related Issues (20)
- Wrong documentation version on gh-pages HOT 1
- No includes in _RestIterable response without hydration
- lang filtering does not seem to work HOT 1
- Attempting to pull more than 10 tweets with search/tweets HOT 1
- Question: how do I acquire `screen_name` using `author_id`?? HOT 1
- DM Example not working HOT 2
- gracefully handling a stalled stream HOT 51
- Upgrade Ads API to version 10 - Deadline March 31, 2022
- Bearer Token Issue with Academic Access in Twitter API V2 HOT 1
- SUBSCRIPTION ENDPOINT NOT WORKING HOT 2
- `expansions` does not work with `author_id` HOT 2
- Twitter Ads API version 10 deprecation - End of life date September 29, 2022
- Add support for `/users/:id/bookmarks` endpoint
- Add support for OAuth2 authentication with access_tokens HOT 1
- Not able to create Webhook with Twitter HOT 2
- Request method fails to attach additional metadata HOT 2
- Authenticating error (api_version='2') HOT 8
- Can't attach media V2 HOT 2
- Publish Tweets w/ v2 HOT 3
- OP_NO_TICKET SSL Option no longer supported by Twitter HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from twitterapi.