mapbox / tilelive-vector

Implements the tilelive API for rendering mapnik vector tiles to raster images.

tilelive-vector

Implements the tilelive API for rendering mapnik vector tiles to raster images.

new Vector(options, callback)

• xml: a Mapnik XML string that will be used to render vector tiles.
• source: Optional, a uri string suitable for use with tilelive.load(). This is fallback source that will be used if no source is found as part of the Mapnik XML parameters.
• base: Optional, basepath for Mapnik map. Defaults to __dirname.
• format: Optional, target output format. Defaults to png8:m=h.
• scale: Optional, Mapnik scale factor. Defaults to 1.

Code concepts

• Backend z/x/y: a request for a raster tile at, say, 3/3/3 does not always mean 3/3/3 is requested from the backend source. The z/x/y requested from the backend source is referred in code by bz/bx/by and generally represent the same or lower zoom level. This allows for features like overzooming, maskLevel tiles, and scale factor adjustment.
• Overzooming: if a tile beyond the maxzoom of the backend is requested, Vector will attempt to render the tile using the parent of the request at maxzoom.
• maskLevel tiles: to avoid requiring many duplicate or empty vector tiles to be generated at high zoom levels, the backend source can specify a maskLevel. If a vector tile is not initially found at some z > maskLevel, Vector will issue an additional request to the backend using the parent tile of of the request at maskLevel. This allows a lower zoom level to "backfill" high zoom levels.
• Scale factor adjustment: the scale argument decrements the backend zoom level such that the requested tile is the visual equivalent (when viewed on the proper dpi device) of its parent counterpart. For example, scale: 2 decrements bz by 1, scale: 4 decrements by 2, and so on.

Backend headers

The Backend and Vector sources set the x-vector-backend-object header to indicate the type of backend source tile loaded. Possible values:

value	description
empty	backend returned no tile
fillzoom	tile from the fillzoom of the backend (backfilled from a lower zoom)
overzoom	tile from the maxzoom of the backend (overzoomed)
default	normal tile

Bonus constructors

There are two additional constructors provided for slightly different workflows for loading vector sources.

Vector.tm2z(uri, callback)

Load a vector source from a uri (filepath or HTTP url) to a tm2-generated tm2z style package.

Vector.xray(options, callback)

Load a vector source with autogenerated data inspection ("xray") styles. xml is not required -- only the uri to a vector source backend is necessary.

tilelive-vector's Issues

Implement backend source as a non-wrapper

Instead, look into using prototype to generate a derivative from the passed source.

?

hi,
?

cache tests

@yhahn I restored a few tests in 983b03f that went missing, but after doing this I'm seeing a few failures that seem to indicate an API change. Is this an easy spot/fix for you? If not I can dig deeper:

  2) cache lockingcache should singleton requests to backend:
     TypeError: Cannot read property '0.0.0' of undefined
      at Context.<anonymous> (/Users/dane/projects/tilelive-vector/test/test.js:341:43)
      at Test.Runnable.run (/Users/dane/projects/tilelive-vector/node_modules/mocha/lib/runnable.js:171:15)
      at Runner.runTest (/Users/dane/projects/tilelive-vector/node_modules/mocha/lib/runner.js:300:10)
      at /Users/dane/projects/tilelive-vector/node_modules/mocha/lib/runner.js:346:12
      at next (/Users/dane/projects/tilelive-vector/node_modules/mocha/lib/runner.js:228:14)
      at /Users/dane/projects/tilelive-vector/node_modules/mocha/lib/runner.js:237:7
      at next (/Users/dane/projects/tilelive-vector/node_modules/mocha/lib/runner.js:185:23)
      at /Users/dane/projects/tilelive-vector/node_modules/mocha/lib/runner.js:205:5
      at process._tickCallback (node.js:415:13)
      at process._tickFromSpinner (node.js:390:15)

  3) cache cached tiles should expire after maxAge:
     TypeError: Cannot read property '0.0.0' of undefined
      at /Users/dane/projects/tilelive-vector/test/test.js:356:55
      at /Users/dane/projects/tilelive-vector/index.js:180:28

Find out why vtile.getTile returns "Tile does not exist"

#125 adds the EMAPNIK eror code within the source.profile() method so that errors can be handled properly in mapbox-upload validate (refs https://github.com/mapbox/mapbox-upload-validate/blob/master/lib/validators/tm2z.js#L19).

NOTE: I think this is a stopgap to a deeper mapnik bug. The style added here was valid in mapnik 3.4.17, but invalid in 3.5.6 because of a Tile does not exist error. This smells like something else is afoot, and while that PR fixes the missing code and should be merged, we should solve the underlying issue asap.

cc/ @GretaCB @flippmoke @springmeyer @mapsam

npm install with node v8.5.0 fails

Looks like this is an issue with the node-pre-gyp installation during the mapnik dependency install

[email protected] install F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik
node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/mapnik/v3.5.14/Release/node-v57-win32-x64.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for [email protected] and [email protected] (node-v57 ABI) (falling back to source compile with node-gyp)

F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" clean ) else (node "" clean )

F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" configure --fallback-to-build --module=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64\mapnik.node --module_name=mapnik --module_path=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64 ) else (node "" configure --fallback-to-build --module=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64\mapnik.node --module_name=mapnik --module_path=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64 )
'mapnik-config' is not recognized as an internal or external command,
operable program or batch file.
gyp: Call to 'mapnik-config --ldflags' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: gyp failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\configure.js:336:16)
gyp ERR! stack at emitTwo (events.js:125:13)
gyp ERR! stack at ChildProcess.emit (events.js:213:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Windows_NT 10.0.15063
gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" "configure" "--fallback-to-build" "--module=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64\mapnik.node" "--module_name=mapnik" "--module_path=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64"
gyp ERR! cwd F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik
gyp ERR! node -v v8.5.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'node-gyp.cmd configure --fallback-to-build --module=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64\mapnik.node --module_name=mapnik --module_path=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64' (1)
node-pre-gyp ERR! stack at ChildProcess. (F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\node_modules\node-pre-gyp\lib\util\compile.js:83:29)
node-pre-gyp ERR! stack at emitTwo (events.js:125:13)
node-pre-gyp ERR! stack at ChildProcess.emit (events.js:213:7)
node-pre-gyp ERR! stack at maybeClose (internal/child_process.js:927:16)
node-pre-gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
node-pre-gyp ERR! System Windows_NT 10.0.15063
node-pre-gyp ERR! command "C:\Program Files\nodejs\node.exe" "F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\node_modules\node-pre-gyp\bin\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik
node-pre-gyp ERR! node -v v8.5.0
node-pre-gyp ERR! node-pre-gyp -v v0.6.38
node-pre-gyp ERR! not ok
Failed to execute 'node-gyp.cmd configure --fallback-to-build --module=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64\mapnik.node --module_name=mapnik --module_path=F:\Projects\Maps\Leaflet\node_modules\tilelive-vector\node_modules\mapnik\lib\binding\node-v57-win32-x64' (1)
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node-pre-gyp install --fallback-to-build
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.

strict:true when loading (validation)

As part of your strictify validation quest I'm seeing another minor semver bump to go for this:

There is currently no method on a node-mapnik mapnik.Map to ask what fontsets are used and what faces are referenced in text/shield symbolizers - however this would be quick to add. node-mapnik already offers a mapnik.fonts() which dumps out all available (registered) fonts by name and where they are on the filesystem. So, it seems reasonable to combine these.

Another approach would be to just try to load the map XML, but in strict mode map.load(<xml>,{strict:true},cb). This mode will throw if any font files are referenced that do not exist..

This should ideally catch the missing font (and also missing image/other assets) cases that you ran into. It will cause some pain for TM2 stylers upgrading who have been sloppy with their styles but the pain will pay off in the long run.

Sound good for

• a new branch, switch to strict mode across the board (no option)
• tests that throw some missing ref styles at it
• new release with semver minor bump

cc @springmeyer

Imagesize option

Add option for adjusting imagesize (Map/Image/Grid) from hardcoded to 256 to several scale sizes (256/512/1024/2048).

Per-request scale factor

Goal: allow scale factor to be set on a per-request/tile basis, much like format.

https://github.com/mapbox/tilelive-vector/blob/master/index.js#L258-L260

• Map currently uses buffer size that is fixed to a single scale factor multiplier. Likely mapnik will need to automatically adjust buffer size based on scale factor at render time (?). https://github.com/mapbox/tilelive-vector/blob/master/index.js#L64

cc @springmeyer

getting tests running on node v0.11/windows

Node v0.11.x works on linux but not windows. For a rainy day:

C:\projects\tilelive-vector\node_modules\tape\index.js:75
        throw err
              ^
Error: ENOENT, open 'C:\projects\tilelive-vector\%5Cprojects%5Ctilelive-vector%5Ctest%5Cfixtures%5Ctm2z%5Cproject.tm2z'
    at Error (native)

https://ci.appveyor.com/project/Mapbox/tilelive-vector/build/1.0.92/job/74msxxwu1gkwlm4w

Raster xray

Raster layers are currently ignored by xray styles. Fix.

Dont underzoom

If you disrespect the advertized minzoom, you'll still get back a vtile with minzoom params here:

https://github.com/mapbox/tilelive-vector/blob/master/backend.js#L62

Leading to all sorts of wackiness.

Use os.tmpdir

http://nodejs.org/api/os.html#os_os_tmpdir

For windows.

cc @rclark

png test failures

It appears that reading from a buffer can mess up imagemagick and causing spurious corrupt image test failures like: https://gist.github.com/springmeyer/67b997ddd28691f38456

symlinks in test fixtures

When installing tilelive-vector from tm2 npm hangs for several seconds and then outputs:

npm WARN excluding symbolic link test/expected/[email protected] -> c.0.0.0.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.1.0.0.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.1.0.1.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.1.1.0.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.1.1.1.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.2.1.1.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.2.1.2.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.3.2.2.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.3.2.3.png
npm WARN excluding symbolic link test/expected/[email protected] -> c.3.2.4.png

Looking back multiple zooms for fill tile

The fillzoom capability of tilelive-vector works well for a tileset with one consistent effective maxzoom, where a fillzoom tile should exist for all child tiles at higher zooms. However, for a "bumpy maxzoom topography" -- where the effective maxzoom of existing tiles is spatially variable -- the ability to load the highest existing parent zoom tile (within a reasonable threshold) would be useful.

A lookback, when specified in the info for a given tileset, would describe the number of zooms that should be looked for, and the highest zoom of these used to overzoom and fill in the missing tile.
For example, when trying to load a z19 tile, where a z17 tile exists, and the lookback is 3:

1. Z19 tile attempts to load, but is not found
2. Z18, Z17, Z16 tiles all all attempted to load
3. The highest zoom found is overzoomed

cc @yhahn

Tests failing after numeral update

Refs adamwdraper/Numeral-js#378

Seeing:

not ok 1233 should be equal
  ---
    operator: equal
    expected: 'Upload size should not exceed 750KB.'
    actual:   'Upload size should not exceed 750KiB.'
    at: Parse.error (/home/travis/build/mapbox/mapnik-swoop/node_modules/tilelive-vector/index.js:425:46)

This was caught by mapnik-swoop as it runs on a daily scrum: https://travis-ci.org/mapbox/mapnik-swoop/jobs/177731433#L1478

[Security] Update request dependency

pre-v2.74 releases of request contain a vulnerability due to an outdated dependency on tough-cookie https://nodesecurity.io/advisories/130

some tests failing with node v0.6.x

As shown by travis: https://travis-ci.org/mapbox/tilelive-vector. But tests are passing with v0.8.x and v0.10.x so not considering this a priority to look at right now.

Densest child tile profiling

Per previous convo:

Add profiler/method to tilelive-vector. Basic algorithm I'm thinking of is:

1. Build a list of "densest" vector tiles from VT source -- grab 4 vector tiles around center at minzoom, choose largest, grab 4 children of that at minzoom+1, recurse...
2. Render each vector tile in the list using the style n times... (what should n be? Will depend on how much results vary by other cpu usage on the system...)
3. Report stats.

Let's build this right into the #profile method.

conflict with internal Tilelive

tilelive is pulled from node_modules subdirectory.
If tilelive is already installed and protocols are added to it. This causes confusion. It needs to be removed in that case...but it becomes a deployment problem when multiple tile protocols are used (mbtiles, vectors...)

permission denied!

I run index.js and this error showed:

permission denied: ./index.js
I added full permission for all file and then this error showed:
./index.js: 1: ./index.js: Syntax error: "(" unexpected

Travis

Preq is mapnik/node-mapnik#173

CI testing on Travis will be a huge help and confidence boost when deploying this production.

Refactor tests

With the introduction of backend...

• Split/add backend source tests
• Add constructor tests

Use bilinear interpolation for rasters

Currently we are using nearest neighbor (the mapnik default) interpolation for the raster symbolizer:
https://github.com/mapbox/tilelive-vector/blob/master/templates/layer.xml#L21

Changing it to scaling="bilinear" will give better looking output tiles, which is especially important if we move towards resolution rounding as detailed here: mapbox/mapnik-omnivore#133

cc @yhahn @springmeyer @perrygeo

Tar vulnerability

We should release a new tag with tar upgraded: https://david-dm.org/mapbox/tilelive-vector

Unable to use with uncompressed vector tiles

Even though makevtile() supports uncompressed PBF data, the code at https://github.com/mapbox/tilelive-vector/blob/master/backend.js#L113 only allows compressed data to be treated as vector tiles. There are several cases when vector tile gets uncompressed and processed by other layers, and they have to be re-compressed just so that tilelive-vector handles it correctly.

Should there be a header or a URI parameter to allow for this?

Test failure: tile format a (0.0.0) as png

Seeing this on master, not sure the cause.

✖ 1 of 54 tests failed:

  1) tiles format a (0.0.0) as png:
     Error: 30.1231 (0.000459649): /Users/mikemorris/Dropbox/Projects/tilelive-vector/test/expected/a.0.0.0.result.png
      at ChildProcess.module.exports (/Users/mikemorris/Dropbox/Projects/tilelive-vector/test/image.js:38:29)
      at ChildProcess.EventEmitter.emit (events.js:99:17)
      at Process._handle.onexit (child_process.js:686:10)

Split out tm2z and xray functionality

I keep on noticing the tm2z and xray functionality and feeling like these should be their own tilelive-tm2z and tilelive-xray modules.

Happy to stop having this itch if @yhahn or @rclark can tell me a good reason they are part of tilelive-vector. But from my perspective tilelive-vector should be about only vtile -> image and other functionality should be separate.

This came up just now because I notice that mapbox-tile-copy depends on tilelive-vector but only for the tm2z support and not for anything to do with rendering images.

/cc @mapsam

missing return cb

Not sure if this is a problem, but why trying to replicate mapbox/mapbox-studio-classic#1047 (callbacks called more than once) I noticed that tl-vector consistently uses return cb except in this one place:

diff --git a/index.js b/index.js
index 587357d..4741c23 100644
--- a/index.js
+++ b/index.js
@@ -414,7 +414,7 @@ function tm2z(uri, callback) {
     var unpacked = false;

     var once = 0;
-    var error = function(err) { if (!once++) callback(err); };
+    var error = function(err) { if (!once++) return callback(err); };

     // Check for unpacked manifest
     exists(base + '/.unpacked', function(exists) {

Failing tests on Windows with node 0.12.7

related? mapbox/tilelive-bridge#52

❌ There are failing tests with node 0.12.7:

https://ci.appveyor.com/project/Mapbox/tilelive-vector/build/1.0.191/job/2de40a9mp4sb0fve#L994

---

✅ Same tests work ok with node 0.10.40

https://ci.appveyor.com/project/Mapbox/tilelive-vector/build/1.0.191/job/pevsqguap2wct5t0#L995

Profiling maps w/ minzoom may miss densest tiles

Since rendering all tiles for an initial profiling step at a high minzoom could take an incredibly long time, profiling a map currently starts around the map center at minzoom, and dives in form there, which could miss dense data outside this limited scope.

Uncompressed mapnik vector tiles are not supported

Passing an uncompressed PBF from a backend's getTile is not supported, see:
https://github.com/mapbox/tilelive-vector/blob/master/backend.js#L106-L112

See #70 for full discussion and mapbox/tiletype#4 for a prerequisite to fix this one

please add a LICENSE file with the content of the BSD license you chose

also a copyright mention (in the README or in the source) is good practice
in case this software is distributed.
Thank you.

Remove all the binds

Not performant.

Backport #149 to v3.9.x and/or v3.10.x

Would it be possible to backport #149 to v3.9.x and/or v3.10.x and release a new package to npmjs?

This is quite an important fix as it addresses a security vulnerability. Migrating to v4.0.0 is a bigger effort that we cannot really do in the short-term.

Thanks

retag against tagged node-mapnik release

I notice that @mikemorris tagged [email protected] with the package.json pointing at master node-mapnik. In this case there should be no concern about this because node-mapnik dev has been stable of late, but in the future we should be careful to avoid tags until all deps are tagged. Sorry of the delay on my part: Tomorrow I'm on track to tag [email protected].

OSM pbf vs. Mapnik Vector Tile confusion (again)

Am I right that the files under test/fixtures/a/ are NOT in the Mapnik Vector Tile format ?
Mapnik Vector Tile files should start with hex 1A 93 (tag version=15, value 2), and indeed they do in the Mapnik Vector Tile testsuite.

But these files under tilelive-vector start with hex 78 9C, and the code even recognizes that 2-bytes sequence to mean "pbf". What's that ? Is it the OSM file format ? Is tilelive-vector supposed to be able to parse MVT encoded vector tiles at all ?

See also mapbox/mapnik-vector-tile#48 and mapbox/vector-tile-spec#20 for additional confusion ...

Switch to loading maps with fromStringSync

I'm worried that mapbox/mapbox-studio-classic#214 is due to async map loading. If this hunch is right then we might be able to workaround the same way as in tilelive-mapnik: mapbox/tilelive-mapnik#58.

But first I'm going to try this: mapnik/node-mapnik#239.

Whatever fix proves successful will also impact whether we load sync or async in tilelive-bridge.

Does 'tilelive' belong to dependencies ?

I've been trying to use tilelive-vector as a tilelive driver (with no much success so far, if there's any example it would be great to see it) and got confused by "tilelive" being included as a dependency of it on "npm install tilelive-vector".

The problem was that I was registering a "test:" protocol to my required tilelive while tilelive-vector uses its own instance of "tilelive" to load the source, thus not finding my own.

Should I register source/backend drivers into the tilelive-vector own instance ?

Note that removing node_modules/tilelive-vector/node_modules/tilelive fixed that issue for me, resulting in tilelive-vector using the tilelive instance in the upper levels.

... I'm sure I'm doing something completely wrong, but anyway decided to file the ticket because tilelive-mapnik (for comparison) does not require 'tilelive' as a dependency

semver tags

Going to start doing semver based tags. The deploy tags were a temporary idea while tilelive-vector was pre-alpha.

Sharing VT instances

I should be possible to share VT instances concurrently as long as access to them is read only.

So, functions that are reentrant, do not mutate data, and therefore should be safe to use concurrently are:

• vt.render
• vt.isSolid
• vt.toJSON
• vt.query
• vt.names
• vt.toGeoJSON
• vt.width
• vt.height

Functions that are not Reentrant (and mutate data) are:

• vt.setData
• vt.parse
• vt.getData
• vt.composite
• vt.clear
• vt.painted setter

Can't override xml-provided source using the constructor opts

It's not possible (or if it is, not obvious how!) to override the source provided in the xml. This would be useful, for example, when using a style generated by tm2, but serving the vector tiles from a local cache on a non-networked device.

With the options passed to the constructor, the source is handled differently from other options, such as scale and format. The xml-provided values for these can be overridden in the constructor opts, but the reverse happens for source - the xml value overrides any source provided in the constructor.

tilelive-vector/index.js

Line 80 in 4969f7a

this._format = opts.format || map.parameters.format || this._format || 'png8:m=h';

I think if a source is provided in the constructor, then it should override the source in the xml, as per the behaviour of scale and format.

Providing a way to change the font of xray default style

When I use tilelive-xray to render a vector tileset, the chinese characters are not properly rendered. Since it hard to find a font that includes all languages all over the world, so providing a way to change the font of xray default style will be better.

Suggestion for improvement: Better samples

It would be helpful for newcomers to the tilelive modules to have a fully working sample they can run. It is a bit daunting getting into this, as there are quite a lot of libraries and moving parts to understand.

I'm struggling to get a working version of converting my own vector tiles to PNGs using this, which is something those familiar with the code can probably do very quickly. Working examples would go a long way to helping in understanding I think.

Thanks

Confusion about tilelive-vector and tilelive-mapnik

tilelive-mapnik is used to rendering tiles with mapnik XML, and tilelive-vector is almost do the same thing as tilelive-mapnik. What is the difference between them?

missing "close" method called by tilelive's copytask

Either https://github.com/mapbox/tilelive.js/blob/master/lib/copytask.js#L94 should check if that method is implemented, or tilelive-vector should add a stub close.

Have encoded image size follow scale factor

The basic idea here is to move away from a fixed 256x256 tilesize and instead have the scale factor of the request affect the output tile size accordingly. So,

request	output size
getTile with scale: 1	256x256
getTile with scale: 2	512x512
getTile with scale: 4	1024x1024

Rough sketch

• Update to node-mapnik 1.4.3 which includes mapnik/node-mapnik#242
• Have Image, Grid, etc. surface sizes reflect the scale factor https://github.com/mapbox/tilelive-vector/blob/master/index.js#L161-L167, Map surface size can prob be dropped to 0 since it should have no effect on renders
• Remove or make optional the backend scale adjustment https://github.com/mapbox/tilelive-vector/blob/master/backend.js#L59-L71
• Legacy mode? Some users (like Mapbox) will need to support the old legacy behavior
• Update all test fixtures, coordinate with a node-mapnik tag

Background on #48

fonts/ should go into tests?

I notice the fonts/ directory and presume it is just for tests. If so we should move it into tests so it is easy to ignore and won't add to the size of mbstudio (though in this case the single fonts is not huge).

Get tests to pass on Windows

The appveyor branch is working towards tackling / proving windows compatibilty. Currently there are a bunch of tests failing, but only five distinct types of failures:

• fails to render image

1) tiles should render i (0.0.0):
516     Uncaught AssertionError: [{"name":"image","extent":4096,"version":1,"features":[{"raster":"","type":0,"geometry":[]}]}] deepEqual [{"name":"image","extent":4096,"version":1,"features":[{"raster":"ffd8ffe000104a46494600010100000100010000ffdb004300060405060504
517      at cbTile (C:\projects\tilelive-vector\test\backend.js:137:32)
518      at C:\projects\tilelive-vector\backend.js:154:13

This is due to an adjustment I made in how a vector tile containing a raster gets JSON.stringified. Previously, value.length was undefined. Still investigating.

• Image size different from fixture

6) tiles should render a@vt (0.0.0):
541     Uncaught Error: Image size is too different from fixture: 11528 vs. 11
542      at imageEqualsFile (C:\projects\tilelive-vector\test\image.js:27:25)
543      at cbTile (C:\projects\tilelive-vector\test\test.js:175:21)
544      at C:\projects\tilelive-vector\index.js:192:28

I modified the threshold for these filesize comparisons, but some image sizes are too dramatically different to ignore.

• Image too different from fixture

20) tiles should render c (2.1.2):
625     Uncaught Error: Image is too different from fixture: 4
626      at imageEqualsFile (C:\projects\tilelive-vector\test\image.js:34:18)
627      at cbTile (C:\projects\tilelive-vector\test\test.js:175:21)
628      at C:\projects\tilelive-vector\index.js:192:28

These occur when mapnik.Image.compare() returns non-zero, and means that the images are significantly different.

• Mapnik is botching some paths

38) tm2z loads a tm2z url:
732     Uncaught Error: ENOENT, open 'C:\projects\tilelive-vector\%5Cprojects%5Ctilelive-vector%5Ctest%5Cfixtures%5Ctm2z%5Cproject.tm2z'
733      at Error (native)

Native code is putting together erroneous path to the .tm2z file. It may be useful to note that these errors do not occur in node v0.10.29.

• svg comparison failure

37) tiles format a (0.0.0) as svg:
727     Uncaught AssertionError: 854232 == 858412
728      at cbTile (C:\projects\tilelive-vector\test\test.js:224:28)
729      at C:\projects\tilelive-vector\index.js:176:24

Just occurs once, this looks like a comparison threshold that needs adjusting.

@yhahn says: looks like a possible CRLF vs LF issue?

---

So far, adjustments have primarily been:

• path.join all the things. Don't do this kind of thing:

var filepath = __dirname + '/folder/' + filename;

• dealing with line-ending when doing string comparisons from fixtures in the file system
• loosening thresholds

cc @springmeyer

svg comparison test failure with node-mapnik v1.1.1

node-mapnik v1.1.x started filtering features that do not intersect with the rendered map bounds when rendering from vector tiles. As such we have an svg test failure:

  2) tiles format a (0.0.0) as svg:
     AssertionError: [60,63,120,109,108,32,118,101,114,115,105,111,110,61,34,49,46,48,34,32,101,110,99,111,100,105,110,103,61,34,85,84,70,45,56,34,63 == [60,63,120,109,108,32,118,101,114,115,105,111,110,61,34,49,46,48,34,32,101,110,99,111,100,105,110,103,61,34,85,84,70,45,56,34,63
      at cbTile (/Users/dane/projects/tilelive-vector/test/test.js:258:28)
      at /Users/dane/projects/tilelive-vector/index.js:203:28