n76 / lacells-creator Goto Github PK

I ran gen_lacells it produced merge.db with a size of 550 MB (I didn't filter out any MCC/MNC)
Then I ran the commented sqlite code in gen_lacells:
.headers on
.mode csv
.output merged.csv
SELECT * FROM cells ORDER BY mcc, mnc, lac, cid;
.quit

then ran: ./merge_duplicate_towers > towers.csv

the resulting towers.csv (415 MB) was smaller than merge.csv (484 MB)

then I ran the rest of commented sqlite commands:
sqlite3 lacells.db <<!
CREATE TABLE cells(mcc INTEGER, mnc INTEGER, lac INTEGER, cid INTEGER, longitude REAL, latitude REAL, altitude REAL, accuracy REAL);
.mode csv
.import towers.csv cells
CREATE INDEX _idx1 ON cells (mcc, mnc, lac, cid);
CREATE INDEX _idx2 ON cells (lac, cid);
VACUUM;
.quit
!

Yet the resulting lacells.db was 691 MB ! Although I notice that this database doesn't have the 'samples' column !

Hello,

You mentioned in README that the script filters out Cell IDs that are outside the US, so I looked at your script to figure out what to remove, so I found those:

INSERT INTO cells SELECT mcc, mnc, lac, cellid, long, lat, -1, min(max(range, 500),100000), max(1,samples) FROM cells_new;

What does that one do ? What is min(max(range, 500),100000) & max(1,samples) ?

DELETE FROM cells WHERE mcc != 310 AND mcc != 311;

I guess that's the filtering you do, right ?

Because actually the LocalGsmNlpBackend 0.9.1 (µgUnifiedNlp without GApps) fails to generate the database (for me), I tried using this script to generate one instead.

While generating I'm getting this happens:
./lacells-creator -i -c262,520 -m
Importing mozilla cell data
./towers_mozilla.csv:8319154: INSERT failed: database or disk is full
Importing opencellid cell data
./towers_opencellid.csv:4441184: INSERT failed: database or disk is full
Error: near line 52: database or disk is full
Error: near line 53: database or disk is full
Filtering rows
Merging cell towers
Error: near line 69: database or disk is full

What can I do to solve this problem ?
And is it a known problem that the LocalGsmNlpBackend (0.9.1) is actually unable to create a database file ?

Thanks :-)

Ran the script and got this error, is this normal? It did generate a .db file in the directory though (less than 1MB in size).

Full output:
Downloading data
rm: cannot remove ‘lacells.db’: No such file or directory
Generating database
off

The final database is called "merge.db" in line 92 and "lacells.db" in lines 83, 84 and 120.
This results in an error when running the script.

Hi,

It's not really an issue, but I would like to contribute my script for lacells.db creation.
It does more-or-less the same thing as lacells-creator does, albeit MUCH faster.

It is simple and has no advanced functionality, but should be enough for most users who only need to create databases.

You can find it here
https://github.com/sobrus/FastLacellsGenerator