seikur0 / pgo-mapscan-opt Goto Github PK

Just saw a post in r/pokemongodev that said the servers changed the radius that you can see pokemon to 70 meters.
If true are you aware and does this still scan at 100m?

https://www.reddit.com/r/pokemongodev/comments/4v73aj/psa_scan_radius_was_changed_from_100m_to_70m/

Scratch that, I just saw your commit where you updated it to set the radius automatically depending on the server response.

Hallo ich habe folgende Exception, wenn ich den Webserver starte.

Kannst du mir da weiterhelfen?

`Exception happened during processing of request from ('192.168.101.254', 17907)
Traceback (most recent call last):
File "/usr/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 651, in init
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 704, in finish
self.wfile.flush()
File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

192.168.101.254 - - [29/Jul/2016 13:44:22] code 501, message Unsupported method ('OPTIONS')
192.168.101.254 - - [29/Jul/2016 13:44:22] "OPTIONS / HTTP/1.0" 501 -
^CTraceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/lib/python2.7/SimpleHTTPServer.py", line 220, in
test()
File "/usr/lib/python2.7/SimpleHTTPServer.py", line 216, in test
BaseHTTPServer.test(HandlerClass, ServerClass)
File "/usr/lib/python2.7/BaseHTTPServer.py", line 599, in test
httpd.serve_forever()
File "/usr/lib/python2.7/SocketServer.py", line 236, in serve_forever
poll_interval)
File "/usr/lib/python2.7/SocketServer.py", line 155, in _eintr_retry
return func(*args)`

Hi there,

First of all thanks for your wonderful scanner. Ive been using it with great success.

I have 2 questions:

1 - I run with 2 different ids in adjacent parts of the town - One id takes 9mins to scan the other 15++mins. I understand that since yesterday the scanning time got prolonged but why the difference between the two ids?

2 - I ve been using ngrok to port the map to mobile but with your scanner it feels it is too much data since it never loads or its extremely slow. Is that normal?

Again I really apreciate your work.

Thank you.

I keep getting the error and retrying when trying to do a google login. Does it actually work?

So i'm positive that I'm looking at the area I'm scanning and I can't seem to see any pokemon in the map. I can see them in the console but nowhere else

With every update I have to re-add some custom bits for my setup e.g log to a SQLite db and also add single scan mode which is useful to run from a mobile device when out and about and you want to know what is nearby.

Would you be willing to add these features? I can set up a pull request if you're interested.

Hi, you're amazing for sharing your awesome work. I apologize for this stupid question, but I can't seem to find an answer to it. I'm a newbie to coding and stuff. I can comprehend only about half of your codes.

I believe I set up usersettings.json correctly with 3 different id's. When I run python main0.py, it will only load one id which is 0. I see that without -id parser it defaults to 0, but what's the point of having multiple profiles if they can run only one by one?

Even if all 3 id's run at the same time, what good does it do if they all search only one spot? All of the id's will end up going through same coordinates.

First, I love this search algorithm!

1. Will you be implementing an after the fact filter into the browser page?
2. Will you be implementing a notification feature (preferably pushbullet)
3. At what range would you recommend using multiple accounts for scanning? I used 1 for an 8 radius scan and it instantly gave me what I wanted. 20 radius scan was only slightly slower(about 15 seconds) a 40 range scan with one account took over 5 minutes(understandably).
4. Do you have a forum/general chat page up for this to keep the github clean(aside from reddit)?
   % Thank you again! I have been so frustrated with the speed of other map scanning programs.

I left my six workers running during the night for data mining. All of them ran flawlessly besides one.
It happened to me before while running multiple workers. The error goes starts and it's not able to recover.

[-] Uncaught connection error, error: Invalid URL u'https:///rpc': No host supplied
[-] Error happened before network request.
[-] Retrying...
[-] Uncaught connection error, error: Invalid URL u'https:///rpc': No host supplied
[-] Error happened before network request.
[-] Retrying...
[-] Uncaught connection error, error: Invalid URL u'https:///rpc': No host supplied
[-] Error happened before network request.

Just a typo on line 141.
except ValueErroras e:
except ValueError, e:

Thanks

Apon execution I get the errors:
File "main0.py", line 541 in
main()
File main0.py line 495 in main
f.close()
UnboundLocalError: local variable 'F' reference before assignment.

Any ideas?

127.0.0.1 - - [01/Aug/2016 21:08:19] code 404, message File not found
127.0.0.1 - - [01/Aug/2016 21:08:19] "GET /res/data0.json HTTP/1.1" 404 -

Not sure what I'm doing wrong here.

any fix to this?

https://imgur.com/a/B1woF
'pip install -r requirements.txt' or if that fails 'pip install --user -r requirements.txt.
It does not help any one method.

Hi,

first of all thank you for the nice application :) Do you have any tips for a python 3 port? As far as I understand a simple 2to3 conversion of every file won't be enough and I don't really want to use Python 2 anymore.

Cheers

I didnt know if this goes here or in pull requests, but I thought that it will be better to click in the google map to change the location, instead of changing it manually in the config file.

Hello there,

we are currently creating a big collection of pokémon spawns/nests and want to ask popular pokemon maps to share their data with us.

for reference:
https://www.reddit.com/r/TheSilphRoad/comments/4uqlet/pokelyzer_setup_instructions_a_tool_for_finding/
https://github.com/Kostronor/pokelector

What we would need are:

• A log file created that adds a new line for each pokémon found.If possible removing hearthbeat updates.
• maybe an option to push all found pokémon over a socket (if the user wants that)

what we need is a data object having these properties:

pokemon_id = int(data['pokemon_id']) # the pokemon number
last_modified_time = int(data['last_modified_timestamp_ms']) # when you received the pokemon from pogo-servers
time_until_hidden_ms = int(data['time_till_hidden_ms']) # the time until the mon is hidden
encounter_id = str(data['encounter_id']) # the encounter id from pogo
spawnpoint_id = str(data['spawnpoint_id']) # the spawnpoint id from pogo
longitude = float(data['longitude']) # longitute of the spawn
latitude = float(data['latitude']) # latitude of the spawn

Sending a json object would be best but other formats are fine too. If you can give me a hint where to look for a pokemon received event, I could submit a pull request.

Hi, first thank you for this improved algorythm really a good idea (others are slow and forced us to scan rather small areas).

So, when executing :
python main0.py -id 0 -r 20 -t 600

It returns the following message :

Traceback (most recent call last):
File "main0.py", line 541, in
main()
File "main0.py", line 431, in main
pokemons = json.load(open('res/'+LANGUAGE+'.json'))
File "C:\Python27\lib\json__init__.py", line 291, in load
**kw)
File "C:\Python27\lib\json__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 380, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 4 column 1 (char 18)

Since I've not any knowledge in python, I don't really understand this message, I've tried to go to the "C:\Python27\lib\json\decoder.py" file or in main0.py at line 4, but it doesn't seems to have any missing "," there.

Any idea where this could comes from?

I've used other map without trouble btw.

This is really great, but it would be fantastic if this script could be more extensible. Right now we have to go in and manually update the code for our user/pass and our google maps key. It would also be great if we more easily could call your script in our own modules:

import PGO_mapscan_opt
while True:
    poke_objs = PGO_mapscan_opt.search(config)
    # Do stuff with poke_objs

where config is yaml or some other data structure:

username = '[email protected]'
password = 'password'
range = 12
location = 'some place'
gmaps_key = ''    # optional

and so on.

print(pokemons[wild.pokemon_data.pokemon_id])

For Nidoran, I got Nidoran? I'm not sure if this applies to other pokemon with maybe special characters in their name? e.g. Mr.Mime etc.

Im trying to track 2 points at the same time, so I put unique_coordinates to true and use the location in the differents ids but don't work.

1 . The collector.py scripts fails at line 150.

File "collector.py", line 150
    print(dot, end='', flush=True)
                  ^
SyntaxError: invalid syntax

If I remove this print command, then I am getting

Connected to server, start sending!

You seem to have no uploadable files.
Wait a bit for the map to accumulate more pokemon.

Transfered 0 pokemon 
sightings!
Transfer done, waiting for data to be send by the os!
Thanks for helping out!

...which isn't right, because I have data from 10 workers collected.

2 . With the recent throttling issues from Niantic (here and here) how does one change the scan delay? Any idea what the change in object refresh means for this, too?

3 . Scan radius changed from 100m to 70m (here). I honestly don't know if this effects the 'centralscan' function. I am not a python guru.

Since a few hours ago (after i went to bed - 9hours ago) the scanners are missing a LOT of pokemon.

I noticed when i was on a Pokestop and saw a ninetails nearby and i wasnt notified. I managed to find and catch the ninetails using a online scanner (trackemon.com).

Then i got home and removed the filter on the scanner and indeed its missing a lot of pokemons.

Check attached pictures:

It has to do with the recent nyantic changes or the changes on your project you made recently?

Thanks

Any one got this issue? I have set the language to English, using google auth.

Traceback (most recent call last):
File "main0.py", line 713, in
main()
File "main0.py", line 675, in main
f.write('{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n'.format(pokemons[wild.pokemon_data.pokemon_id],wild.pokemon_data.pokemon_id,spawnIDint,wild.latitude,wild.longitude,(wild.last_modified_timestamp_ms+wild.time_till_hidden_ms)/1000.0-900.0,wild.last_modified_timestamp_ms/1000.0,org_tth/1000.0,wild.encounter_id))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 8: ordinal not in range(128)

Can you please clarify this parameters a bit more:
"_comment2": "If unique_coordinates is set to false, profile coordinates will be overwritten with standard_coordinates.", "unique_coordinates": false,
and
"_comment5": "Set to true to overwrite the coordinates for id 1 to 6 to scan around the id 0 pseudo-hexagon.", "centralscan": false,

Situation 1: I have multiple accounts, and I want to scan a continuous area.
-What are the settings? "false","false"
-Should I leave the workers with lat and lng as "0" on the profiles settings? Also for id0?
-Should I expand the radius?
-Or it "attaches" around a new hex to the side of the id 0 one? ie: id 1 top left, id 2 top top, id 3 top right...

Situation 2: My area has some gaps or I just want to place my workers far away from each other.
-What are the settings? "true","false"
-What should I type in the standard coordinates? Is this the ones the browser will lock at the start?
-Should I type the lat and lng for each one on their profiles?
-Do I need to run multiple instances of the script for each worker? main0.py -id 0, main0.py -id 1 ,main0.py -id 2...

I do know some of this answers, however, would be good to have them better explain on the readme. I could write a step by step for this if you want.

Best

Hallo seikur0,

Klasse Arbeit die Du hier gemacht hast, ist viel schneller und genauer als die andere Version.

Könntest du dieselbe Funktionalität integrieren wie bei AHAAAAAAA/PokemonGo-Map,
d.h. neuer Ort über die Map-Seite und ein forward nach draußen, sodass ich im Felde mit dem Handy darauf zugreifen kann??

Sorry kann dies leider nicht selbst und weiß auch nicht ob es überhaupt geht, da ich nicht programmieren kann.

Have had this happen very infrequently (twice out of maybe 30 or so times I've restarted threads) upon starting a thread.
It logged in but before scanning,

Traceback (most recent call last):
File "c:\mapscanopt\main0.py", line 721, in
main()
File "c:\mapscanopt\main0.py", line 599, in main
settings.ParseFromString(response.returns[4])
File "c:\python27\lib\site-packages\google\protobuf\internal\containers.py", line 64, in getitem
return self._values[key]
IndexError: list index out of range

Trying again without changing anything works.
Just thought you'd want to know, if you want to implement some debugging or error catching for it.

When you merged my pull request you left 2 lines in that void the change:

        if (settings['unique_coordinates']==false) {
        LAT_C=settings['standard_coordinates']['lat'];
        LNG_C=settings['standard_coordinates']['lng'];
    }else{
        LAT_C=settings['profiles']['0']['coordinates']['lat'];
        LNG_C=settings['profiles']['0']['coordinates']['lng'];
    }
    **LAT_C=settings['standard_coordinates']['lat'];
    LNG_C=settings['standard_coordinates']['lng'];** 

I was so focused on spelling errors or a missing comma I almost missed this

I am running 2 instances now. So 2 sets of workers. The first is id 0, and has 3 accts, the second is id 1, and has 2 accts.

id 0 is returning results, and removing empty locations.
id 1 is returning no results and has removed every location it has scanned.

1. Where is this list of removed locations saved if anywhere?
2. Let's pretend that nothing spawns in said location on this scan, what about on the next scan when something did spawn?
3. Is it possible that running 2 instances with 2 ids each with their own group, that the second group will return no results?

I ask this because I am running a scan of the same area as id 1 (with a different account on an app on my phone) and it has found plenty of pokes.

I just downloaded the newest zip file and edited my account login & pw in the usersettings.json file but if i run the scanner like i did on the older version, i just pops a black cmd/phyton screen and immediately closes it.

Hi,

everytime i need to modify the index.html (do i need this? or is it compatible?)
I added the "Locate Me" Button..

Can u merge this in your file?

Second Request: seperate the user accounts :) maybe database driven?

I want to make LAT_C and LNG_C to be set from the command line arguments, so I set LAT_C and LNG_C to be 0 and added in 2 additional argument parameters to take lat and lng, then I did LAT_C = args.lat and LNG_C = args.lng and the values are obtained correctly. However, it seems like init_location is still using 0 for lat_c and lng_c for some reason. Why is that? :(

Hallo seikur0,

wie kann ich in der Map mehrer ID´ßs auf einmal anzeigen lassen.

Wenn ich mit python main0.py starte wird mir immer nur die ID0 Koordinaten mit Leben befüllt.

Habe:
"unique_coordinates": true,
"centralscan": false,
"load_ids": [0,1],

Ich bekomme nur Inhalt für ID1 wenn ich python main0.py -ID 1 starte.

Oder geht es nicht die Map mit 2 Koordinaten zu befüllen?

Hey, this is maybe a simple question for you. Multithreading is awesome but is it possible to allow more than 5 threads? if i wanna add one login more in the .json file and run the script it just closes immediately. Thx for reply and sorry for bad english

example. if i add one more of this the script wont work.

{
"id": 0,
"type": "ptc",
"username": "xxxx",
"password": "xxxx",
"coordinates":
{
"lat": xxxx,
"lng": xxx,
"alt": 0
}
}

After an hour or two of peaceful scanning this started flooding my cmd prompt.

using 10 accounts
scaninterval set to 300
range set to 20

that's all I tweaked.

I stay near a big lake and scanning takes forever because there are no pokemon on water. I assume after the first run, it will go a lot faster because it will skip the non-responding coordinates.

I'm thinking to grab all of these bad points on map and store them somewhere and modify the codes, so the program doesn't go through these areas every time I run it. Can somebody guide me in the right direction for this modification?

This is going to be a long list of coordinates, since the lake is about 2 miles long and has a bad shape to it.

EDIT: I thought this was happening because of the water but this also happens at other areas. I have both python 2.7 and 3.x installed. Could this be why this is happening?

@seikur0: Looking at the line in this file, you've left a comment saying it's a transaction id.

Care to elaborate? Asking for AeonLucid/POGOProtos#94 :)

Hey seikur0, I really like this implementation because it's a pretty accessible codebase.
I still have some questions regarding your implementation

• What exactly is a heartbeat (what does heartbeat()) do?
• What is an ll?
• What does this loop in main() do?

for a in range(1,HEX_NUM+1):
        for i in range(0,a*6):

• What does getNeighbors do?

Thanks

Hey, this is more of a feature request, but as I loaded up the data to make a heat map, I noticed the time and have no idea what it correlates to per my local time. Would it be possible at some point to make it so we can have an option in the config file to set a time zone or at least time display prefs?

I've been testing with different scanner mode for multiple ids. They do scan and collect the data.
However, only id0 shows on the web map.
I've run multiple instances main0.py -id0, main0.py -id1, main0.py -id2....

Traceback (most recent call last): File "main0.py", line 24, in <module> from pushbullet import pushbullet ImportError: No module named pushbullet

I tried with the settings to true (with credentials) and false on usersettings.json.
The script works if I comment the line on main0.py

Thanks

I've been running the scanner and getting just a few Pokemons. I even reduce the radius to 10.
Are we going to have to run multiple workers for a small area?

Running this with pyhton2.7 on windows i get the following exception:

[+] login for ptc account:
Traceback (most recent call last):
File "main0.py", line 547, in
main()
File "main0.py", line 434, in main
do_login()
File "main0.py", line 225, in do_login
login_ptc(li_user,li_password)
File "main0.py", line 192, in login_ptc
ticket = re.sub('.*ticket=', '', r.history[0].headers['Location'])
IndexError: list index out of range

Code seems to work fine, but I get this error in between each scan. Any ideas if it is important or can I just ignore with it?

Got the latest update, and the second scan is returning a ton of heartbeat errors.

Didn't see these last night, not sure if it's affecting my scan quality or what. Is there a log file which we can peruse?

This is popping up. Also can't CTRL C out of it.

figured it out sorry!

main0.zip

Looks like the time is static, it's not counting down at all. I need to refresh the site for it to work.

Just got the last version to work. Everything worked; however, this message showed constantly in my terminal window.
[-] Response error, status code: 52, retrying in 1 seconds
[-] Response error, status code: 52, retrying in 1 seconds

If I change the backup size to 0 returns an error.

So no matter how many accounts I have, I always get this error after the last account is logged in, even if it seems previously successful?