pvatala / geomodel

Automatically exported from code.google.com/p/geomodel

geomodel

Exported from code.google.com/p/geomodel

geomodel's Issues

Combining proximity_fetch and order("-modified") is not working here

What steps will reproduce the problem?
a = A.proximity_fetch(A.all().filter("modified ",
 timeline).filter("published =", True).filter("modified <=",
 bookmark ).order("-modified") ,db.GeoPt(lat, lon),max_results=PAGESIZE
 +1, max_distance=m)

What is the expected output? What do you see instead?
order("-modified") should sort by time. Results come out seemingly random 
sorted.

What version of the product are you using? On what operating system?
Google App Engine production server + geomodel 

Please provide any additional information below.
I'm trying to order by date using db.GeoPt to store google maps coordinates 
with GAE and geomodel I can map and match Sao Paulo and can't sort by time 
linked http://forum.koolbusiness.com/li?lat=-23.33&lon=-46.38

Original issue reported on code.google.com by [email protected] on 21 Sep 2010 at 8:28

Attachments:

• Screenshot.png

More logging

Logging using Python's standard logging module.

Original issue reported on code.google.com by [email protected] on 5 May 2009 at 8:44

Documentation for how bounding box and proximity queries work in geomodel.py

See TODO(romannurik) item in geomodel.py

Original issue reported on code.google.com by [email protected] on 27 Apr 2009 at 6:40

geomath.py distance calculation sometimes produces math Value error

What steps will reproduce the problem?
1. compute a distance between two identical points at lat:48.856668 lon:
2.350987

What is the expected output? What do you see instead?
geomath.py", line 44, in distance
    math.cos(p1lat) * math.cos(p2lat) * math.cos(p2lon - p1lon))
ValueError: math domain error

This is because it seems that 1.0000 may be > 1.  

What version of the product are you using? On what operating system?
MacOS X snow leopard, latest version as of today

Please provide any additional information below.
Solved the issue by doing
  return RADIUS * math.acos(max(min(math.sin(p1lat) * math.sin(p2lat) +
      math.cos(p1lat) * math.cos(p2lat) * math.cos(p2lon - p1lon),1),-1))

Original issue reported on code.google.com by [email protected] on 21 Dec 2009 at 7:16

If locations of many models are the same, duplicates are possible in in proximity_fetch

What steps will reproduce the problem?
1. Create a database where you have many nodes with the same lat lon, and 
proximity_fetch can 
end up returning dupes due to a bug in _merge_results_in_place because ordering 
becomes 
arbitrary.

FIX: 

    def _merge_results_in_place(a, b):
        def my_cmp_fn(x,y):
            result = cmp(x[1],y[1])
            if result == 0:
                return cmp(x[0].key(),y[0].key())
            return result

      util.merge_in_place(a, b,
                        cmp_fn=my_cmp_fn,
                        dup_fn=lambda x, y: x[0].key() == y[0].key())

Original issue reported on code.google.com by [email protected] on 27 May 2010 at 3:14

Google Application Engine could not build index of "location_geocells and datetime"

I tried to use geomodel with my application and I found this in my index.yaml

- kind: Coupon
  properties:
  - name: location_geocells
  - name: created_at
    direction: desc

It works on my test environment, but when I uploaded it to Google
Application Engine, GAE could not finish building this index for a long time.

I think maybe this is because created_at is a date-time property.

Original issue reported on code.google.com by [email protected] on 25 Apr 2010 at 11:34

Incorrect optimization

Sometimes, a search will fail to bring up otherwise valid results because of 
this optimization in geomodel.py (async branch):

      # Stop the search if the next closest possible search result is farther
      # than max_distance or, if we have max_results results already, farther
      # than the last result.

this optimization is correct, but evaluated after expanding cur_geocells, which 
means that the algorithm wrongly assumes that any additional results are 
farther away than the new expanded cur_geocells and not just farther away than 
the already searched cur_geocells. See attached patch for possible fix, which 
moves the expansion logic below the optimizations.

Original issue reported on code.google.com by [email protected] on 25 Aug 2010 at 11:34

Attachments:

• patch.txt

Proximity queries need better optimization

Proximity queries currently can balloon out in query time if max_distance 
is unbounded.

Another point of optimization is in the first few geocells they look at.. 
for example, this is a trace of the first few cells looked at by a 
proximity query:

9a9c04895e7c9
9a9c04895e7c
9a9c04895e7
9a9c04895e
9a9c04895
9a9c0489
9a9c048
9a9c04
9a9c01

This can probably be optimized in some way.

Original issue reported on code.google.com by [email protected] on 27 Apr 2009 at 6:39

Proximity queries don't work well near boundaries

What steps will reproduce the problem?
1. search for wausau (near minneapolis, approximately lat 45, long -90)
2. zoom out progressively

What is the expected output? What do you see instead?
- The same schools should be shown (or nearly the same)
- Instead, they vary wildly

I don't know whether the bounding box query suffers from the same problem

Original issue reported on code.google.com by [email protected] on 7 Aug 2009 at 2:05

Proximity Query very slow

I am not sure if this is a defect or not but the proximity query is very very 
slow. I have around 2700 records in the database with GeoCells calculated. 

When I run the search query, with 5000 proximity distance and 20 max results, 
it takes around 15-20 seconds to return the results. 
On DEV server, one auto generated index was added and I have added some more 
indexes but to no avail. 

I have not tried the code in Async branch, I will try with that but any help 
would be highly appreciated.

Thanks in advance.

Original issue reported on code.google.com by [email protected] on 25 Jun 2010 at 12:46

proximity_fetch misses single entity, starts showing up when another is added

What steps will reproduce the problem?
1. do a proximity fetch
for: an entity with location (45.0616794,-83.4327528)
from center (42.270872, -83.726329)

What is the expected output? What do you see instead?

no results even with radius set to 0, or something very high (the two points 
are ~200km apart)

What version of the product are you using? On what operating system?
top of tree r22. mac osx

This was reproduced using nosetests with app engine integration.

One thing that is interesting is that adding another point for that entity 
makes it so both show up; the bug only seems to be in place when the 
point is the only entity in the datastore

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 11 Feb 2010 at 1:13

No results are returned from my search

What steps will reproduce the problem?
1.copied source code and added test data to appengine PublicSchool model.
2.run app and search for results in database
3.

What is the expected output? What do you see instead?
I get a success, line 444 in front.js, but no results are returned from my 
sample data.

What version of the product are you using? On what operating system?
Windows XP, geomodel svn trunk

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 23 Jun 2010 at 10:52

Allow paging for proximity queries

Currently, proximity queries return one result set and are finished--no 
paging can be done. There needs to be a way to progressively ask for more 
results.

One idea is to have a 'minimum distance' parameter to proximity_query so 
that the query for 'page 2' can use a minimum distance slightly smaller 
than the distance of the last page 1 result.

Original issue reported on code.google.com by [email protected] on 27 Apr 2009 at 6:42

Please document relationship between resolution and box size

I want to create my own simple query to check if a point is in a box. Can you 
please document the relationship between the resolution (i.e. the length of the 
Geocell string) and the box size (even approximately) e.g. a 13 character cell 
is 100 meters x 100 meters, a 12 char is 400 x 400m etc.

Thanks for the great project!

Original issue reported on code.google.com by [email protected] on 4 Aug 2010 at 11:49

Support bounded box and proximity fetches on GqlQuery

Currently bounding_box_fetch and proximity_fetch only support regular
Query, and not GqlQuery. We should support the latter somehow.

Original issue reported on code.google.com by [email protected] on 3 Sep 2009 at 4:13

18 RunQuery calls in proximity_fetch

Is this normal?

I currently have not entities matching query criteria, and am expecting zero 
results, but I'm setting max_results to 30, and max_distance to 3000 meters.

/_ah/stats shows that RunQuery is getting called 18 times.

Original issue reported on code.google.com by [email protected] on 16 Feb 2011 at 8:21

Support multiple geometry types

We should support lines and polygons at some point via a 'geometry'
property on GeoModel, and also add line and poly to geotypes; 'location'
should, in the future, be a convenience property for setting the geometry
to a GeoPt or geotypes.Point

Consider porting geometry code from the 'geojs' or 'Shapely' projects.

Original issue reported on code.google.com by [email protected] on 15 Jul 2009 at 6:57

geomodel's update_location calls geocell inappropriately

What steps will reproduce the problem?
1. running entity.update_location()


What is the expected output? What do you see instead?
Although no error is returned for certain lat/lon pairs, because geomodel
is passing just a latitude instead of the point object expected by geocell,
you may see an error for certain inputs.  Whether an error is raised or
not, the computation seems incorrect.

What version of the product are you using? On what operating system?
n/a

Please provide any additional information below.
update_location should call compute with a point object rather than
(lat,lon).  Attached is a patch.  Side note: my personal patch also allows
update_location to accept an optional resolution that gets passed to
compute if there's interest in that.

Original issue reported on code.google.com by [email protected] on 3 Jun 2009 at 9:27

Attachments:

• geocell_geomodel_jlivni.patch

more filter automagic requested

It would be nice to have query extended like this:
GeoModel.all().filter("location within",bbox).filter("location 
distance",(geopt,dist)).fetch(count)

Attachement implements .filter("location within",bbox)

in GeoModel there should be

@classmethod
def all(cls,**kwargs):
  return GeoQuery(cls,**kwargs)

Original issue reported on code.google.com by [email protected] on 13 Nov 2010 at 11:58

Attachments:

• geoquery.py

make update_location automagically

please, consider DerivedProperty decorator at 
https://github.com/Arachnid/aetycoon
this allows very simply to describe:

@aetycoon.DerivedProperty
def geocells(self):
 "calculate geocells here instead of update_location"

This avoids the need to call update_location before saving.

Original issue reported on code.google.com by [email protected] on 13 Nov 2010 at 6:58

whole-world geocell required

when request is made over all the world,
16 geocells are in search.

combined with any 2 another searches, this brings 32 requests - over the limit 
of maximal queries per request allowed...

Original issue reported on code.google.com by [email protected] on 13 Nov 2010 at 11:40

GeoModel 0.0.8 for java seems don't work...

Hello,

I'm using GeoModel API for java in version 0.0.8 for my GAE project.
In my google app engine database, I have 10 000 entities which are coord. GPS :
    @Persistent
    @Geocells
    private List<String> geocells = null;

    @Persistent
    @Latitude
    private Double latitude = null;

    @Persistent
    @Longitude
    private Double longitude = null;

Each entity has a list of geocells built by : "geocells = 
GeocellManager.generateGeoCell(pt);"
You can see the screen capture in the dataViewer.PNG file...

I want to lauch a request to find all entities in a radius of 40km for example. 
The result is always empty !!!! :-( :-(.


Example of code in a servlet :

int radiusInKm = 40;
int nbMaxResults = 10000;
Point pointOfMarseille = new Point(43.295262d, 5.374049d);

final PersistenceManager pm = PMF.get().getPersistenceManager();

try {
    pm.currentTransaction().begin();

    GeocellQuery baseQuery = new GeocellQuery();

    // we want all entities in the radius 
    long start = System.currentTimeMillis();
    List<MyEntity> res = GeocellManager.proximitySearch(pointOfMarseille, nbMaxResults, radiusInKm * 1000.0d, MyEntity.class, baseQuery, pm);
    long end = System.currentTimeMillis();

        LOGGER.info(res.size() + " element(s) found(s) in a radius of " + radiusInKm + " km around of Marseille city for maxResults : " + nbMaxResults + " (in " + (end - start)/1000.0d + " sec.)");

    pm.currentTransaction().commit();
} catch (Exception e) {
    pm.currentTransaction().rollback();
    throw new RuntimeException(e);
} finally {
    if (!pm.isClosed()) {
        pm.close();
    }
}

My queries return always empty and I don't understand.
Can you help me ?

Thanks a lot,
F.

Original issue reported on code.google.com by [email protected] on 2 Aug 2011 at 10:13

Attachments:

• dataViewer.PNG

consulitaion/explanation needed

I cannot find better way to contact developers.
The question is - why you geogrid is set up to be 4x4?
Why not 2x2 or 8x8?
How does it impact on performance or precision/performance balance.

Some of coords/hash conversion tools are marked as "working only on 4x4 grid". 
Will the whole system work if to change grid dimentions?

Original issue reported on code.google.com by [email protected] on 18 Nov 2010 at 8:41

Support for automatically updating geocells on Model.put()

It would be relatively simple to override the inherited put() method to
call update_location() and thus automatically update the geocell data every
time an entity is updated.

I realize this may not be ideal for all (or even many) uses of the
geomodel, but it could be turned off by default, and activated by setting a
class attribute to true, as in the attached diff.

Original issue reported on code.google.com by evan.kaufman on 8 Jan 2010 at 5:29

Attachments:

• geomodel.patch

Typo in geotypes.py leads to unexpected behaviour

Steps to reproduce the problem:
- Assign a value to "east" property of the Box class

Problem:
- Due to a typo in the code _set_east() assigns longitude value to a latitude 
attribute. This causes unexpected behavior, naturally.

I discovered this while trying to limit bounding box searches to certain region 
and it's obviously just a copy-paste typo in the code. See attached patch for a 
fix.

Original issue reported on code.google.com by [email protected] on 21 Jan 2012 at 9:19

Attachments:

• geotypes.py.patch

async branch fails if no results on unordered bbox fetch

If using the new async branch, and you call bbox_fetch for a location where 
your query will return no results, you get a divide by 0 error

Simple fix attached

Original issue reported on code.google.com by [email protected] on 7 Oct 2009 at 2:44

Attachments:

• no_results.patch

Unit tests for geocell, geomath, geotypes, util

We need to add unit tests for these core modules:

geocell, geomath, geotypes, util

Original issue reported on code.google.com by [email protected] on 14 Jul 2009 at 7:54

Port geomodel to the upcoming ndb datastore API

Google App Engine looks likely to release an improved datastore API called ndb 
[http://code.google.com/p/appengine-ndb-experiment/] in the not-too-distant 
future as part of the official SDK.

ndb offers a number of benefits - particularly relating to performance and 
executing queries asynchronously.

My app already uses ndb instead of db, and I also plan to use geomodel. For now 
I'm going to port the current version of geomodel to ndb myself, but it would 
be awesome if the geomodel project could be made ndb-compatible so that 
developers like myself can benefit from future updates.

Original issue reported on code.google.com by [email protected] on 5 Sep 2011 at 8:26

Feature requesting country code proximity

What steps will reproduce the problem?
1. Input a country code eg IN for india
2. There's no way to match a query
3. Match manually ie 
http://www.koolbusiness.com/li?lat=21.1894428&lon=77.3783789&cc=IN

What is the expected output? What do you see instead?
See above

What version of the product are you using? On what operating system?
Latest version controlled

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 25 Jul 2010 at 12:07

Spherical Approximation of Earth's Radius

`geomath.py` defines its spherical approximation of Earth's radius as 6378135 
meters (6378.135 km).

According to Wikipedia, the above radius is the actual equatorial radius of the 
Earth (per the WGS-84 ellipsoid model). 

It seem more appropriate for `geomath.distance()` to use an average radius 
value, since the above applies only at the equator, where it is a maximum 
value. The IUGG (according to Wikipedia) defines the mean radius of the earth 
as 6371.009 km. 

http://en.wikipedia.org/wiki/Earth_radius#Mean_radii

Therefore, I propose changing `geomath.py` to instead define:

`RADIUS = 6371009`

(However, there are several different ways to calculate "mean radius" of the 
Earth; I'm not 100% certain that this one is the "most correct" approximation 
for Great Circle calculations, but it is definitely "more correct" than the 
equatorial radius.)

Original issue reported on code.google.com by [email protected] on 7 Mar 2012 at 5:39

Investigate whether one location_geocells property is actually slower than many location_geocell_N properties

Originally, geocells were stored on each entity in one StringListProperty. 
This was changed in favor of a large number of location_geocell_N 
properties (where N=1, 2, ..., 13). 13 is currently MAX_GEOCELL_RESOLUTION.

Although this probably improves query-time performance slightly, are the 
performance gains worth the hassle of introducing 13 internal properties on 
each entity, as well as requiring developers to have 13 new indexes in 
index.yaml for each new type of query?

Original issue reported on code.google.com by [email protected] on 27 Apr 2009 at 6:51

Patch for /trunk/geo/geotypes.py

For US boundary with north=71.5388&east=-66.8850749&south=18.7763&west=170.5957 
bounding_box_fetch was returning no results. With this added it would account 
for antimeridian crossing similar to geojs.

Original issue reported on code.google.com by [email protected] on 22 Jan 2011 at 5:54

Attachments:

• geotypes.py.patch

Async branch doesn't return all results because of flaw in async_in_query_fetch()

What steps will reproduce the problem?
Perform a proximity_fetch() which should return 2 results.


What is the expected output? What do you see instead?
Except to get 2 results.  Instead, I only get 1.

What version of the product are you using? On what operating system?
The async branch, GAE 1.3.3.

Please provide any additional information below.

The issue is in async_in_query_fetch() in util.py, specifically the _numkeep 
lambda:
_numkeep = lambda arr: int(math.ceil(len(arr) * 1.0 / total_results))

This returns 1 when passed in the list [[result1, result2], []], should return 
2.

My quick and dirty work around is to skip the whole map/reduce step if 
total_results <= 
max_results:

    if total_results <= max_results:
      return [ entity for entities in entities_by_value for entity in entities ]
    else:
      #this _numkeep is broken:  returns 1 for [['a','b'], []]
      _numkeep = lambda arr: int(math.ceil(len(arr) * 1.0 / total_results))
      entities_by_value = [val_entities[:_numkeep(val_entities)]
                           for val_entities in entities_by_value]
      return reduce(lambda x, y: x + y, entities_by_value)

Original issue reported on code.google.com by [email protected] on 24 Apr 2010 at 3:42

Async returns max_results, but not centered around query location.

What steps will reproduce the problem?
1. Use async branch to search a large database, using large max distance.

What is the expected output? What do you see instead?
max_results results are returned, but they are not clustered around the search 
location. They may be offset towards one compass quadrant (NE, SE, SW, NW)

What version of the product are you using? On what operating system?
Async branch, patched for issues 13, 16, 23, 33.

Please provide any additional information below.
This behaviour is caused by over optimisation. max_results are found and 
returned before searching adjacents geocells of equal resolution for closer 
results.

Eliminate this behaviour by changing geomodel.py from:

if len(results) >= max_results:
        if DEBUG:
          logging.debug(('GeoModel Proximity Query: '
                         'Wanted %d results, ending search') %
                        (max_results,))
        break

to:
if len(results) >= max_results and len(cur_geocells) == 4:
        if DEBUG:
          logging.debug(('GeoModel Proximity Query: '
                         'Wanted %d results, ending search') %
                        (max_results,))
        break

Original issue reported on code.google.com by [email protected] on 5 Nov 2012 at 12:45

importing help

I'm a noob to appengine so I am somewhat confused as to how to import this 
module to get 
geo.GeoModel to work in my model.

Original issue reported on code.google.com by [email protected] on 4 Jul 2009 at 12:47