mapillary / mapillary-js Goto Github PK

View Code? Open in Web Editor NEW

423.0 38.0 80.0 59.4 MB

Interactive, extendable street imagery map experiences in the browser, powered by WebGL

Home Page: https://mapillary.github.io/mapillary-js

License: MIT License

JavaScript 0.21% CSS 1.32% TypeScript 98.45% Dockerfile 0.02%

imagery webgl typescript mapillary spatial-visualization computational-geometry street-imagery 3d viewer renderer

mapillary-js's Introduction

MapillaryJS

MapillaryJS is a client-side JavaScript library for interactive, extendable street imagery map experiences on the web. It takes spatial, semantic, and texture data and renders it using WebGL. MapillaryJS can be customized with camera controls, user interactivity, and data providers and it can be augmented with geospatial rendering, animation, and content placement.

Installation and Usage

To start using MapillaryJS with data from the Mapillary platform, you need an account. When signed in, you need to register an application to get a client access token. When providing your own data, no access token is needed.

ES6 bundler

Install the package via Yarn (or npm).

yarn add mapillary-js

Use a CSS loader or include the CSS file in the <head> of your HTML file.

<link
  href="https://unpkg.com/[email protected]/dist/mapillary.css"
  rel="stylesheet"
/>

Include the following code in your JavaScript file.

import { Viewer } from "mapillary-js";

const viewer = new Viewer({
  accessToken: "<your access token>",
  container: "<your HTML element ID>",
  imageId: "<your image ID for initializing the viewer>",
});

TypeScript

Install the package via Yarn (or npm).

yarn add mapillary-js

Use a CSS loader or include the CSS file in the <head> of your HTML file.

<link
  href="https://unpkg.com/[email protected]/dist/mapillary.css"
  rel="stylesheet"
/>

Include the following code in your TypeScript file.

import { Viewer, ViewerOptions } from "mapillary-js";

const options: ViewerOptions = {
  accessToken: "<your access token>",
  container: "<your HTML element ID>",
  imageId: "<your image ID for initializing the viewer>",
};
const viewer = new Viewer(options);

CDN

Include the JavaScript and CSS files in the <head> of your HTML file.

<script src="https://unpkg.com/[email protected]/dist/mapillary.js"></script>
<link
  href="https://unpkg.com/[email protected]/dist/mapillary.css"
  rel="stylesheet"
/>

Add a container to the <body> of your HTML file.

<div id="mly" style="width: 400px; height: 300px;"></div>

The global UMD name for MapillaryJS is mapillary. Include the following script in the <body> of your HTML file.

<script>
var { Viewer } = mapillary;

var viewer = new Viewer({
  accessToken: "<your access token>",
  container: "mly",
  imageId: "<your image ID for initializing the viewer>",
});
</script>

Documentation

Code of Conduct

Facebook has adopted the Contributor Covenant as its Code of Conduct, and we expect project participants to adhere to it. Please read the full text so that you can understand what actions will and will not be tolerated.

License

MapillaryJS is MIT licensed.

mapillary-js's People

Contributors

Stargazers

Watchers

Forkers

bzhang1670 kod3r gitter-badger charlesxuuu boguscurry caboosey liviadegrossi npmcdn-to-unpkg-bot bermi raykendo neurostep nirabpudasaini sumarin kublaj originalsin gijs balhalees abbe98 lekstuga donrv camillaisaksson eduardoschoepf ryfx hadmap geoengtechel 4ker cuulee luciplugaru mdiener21 dephora metrowave jakesmarter keplergt leewoody bingoxyb synergy-robotics-a-b rlucas1 worliaro matrixy todkang salusutis openhistorymap h2atecnologia aninggo purpletortoise-choi mapwaylabs jingasan flywave bankbiz fadedweiss ufmg-me thangqd radzul avalone007 powerscans bezerras-org-travels alhuangfb graffity-technologies pcapelo svenkreiss echosystemmark nighthack36 vojvod yahia1018 straybro aniket-kale m8e xcugasbackend1 mmtxcp rpinnama curtis18 nejcfrankfri osm-hackfest-ph zrnwsy dhairyamodi2 luvnft benjamin-loison sushil642 arghyachatterjee shotsan

mapillary-js's Issues

Create spy function to handle fake tiles

Fake tiles woul be generated on every /h API request.

Indicate loading in coverui on click

Before disable animation starts and after button is clicked (this is the waiting time for initiating the background library).

Image size handling

Make it possible to either set the image size in the viewer constructor or handle it internally based on the execution environment.

Make 640 the default size.

Add GlRenderer and DOMRenderer

After running with the UI concept for a while its pretty clear that we need a better place for rendering both for gl and DOM content. I suggest we create two renderers a GlRenderer and a DOMRender and instead of seeing the UIs as endpoints they will have access to a service where they can push data to render.

                            +----------------+
                            |                |
                            | State Context  |
                            |                |
                            +-------.--------+
                                    .
                                    .
                                    .
                                    .
                            +-------.--------+
                            |                |
                            |      UI        |
                            |                |
                            +----------------+
                                 -/   \--
                              --/        \--
                           --/              \--
                         -/                    \--
                +-------o-------+        +--------\-------+
                |               |        |                |
                |  DOMRenderer  |        |    GlRenderer  |
                |               |        |                |
                +---------------+        +----------------+

The renderer will have a rx pipeline as input for the Renderers, using the AnimationFrame as scheduler for the GlRenderer is probably a good idea.

URL util class

Create a class for the URLs used in the library to avoid duplication.

Rename all private attributes to start with _

Also make sure to share as little as possible from object instances.

Create a DirectionUI

A UI that will both work with panoramas and normal photos. It will consist of direction arrows (panoramas and perspective photos) and turn left/right arrows (perspective photos). This will be a default UI. For the time being the experience will be similar to what is at mapillary.com.

Trajectory type on state

It should be possible to set the trajectory type, e.g. Linear and Spline, on the state.

Update PlayerUI to match state

I created a naive PlayerUI. However Im not sure how to handle the state API. The solution might be to remove passed nodes in the trajectory per default.

Fire movestart and moveend events

Subscribe to state and fire movestart and moveend events appropriately.

The events should be based on the trajectory length, the current trajectory index and the alpha value.

Create JourneyUI as demo

Input is sequences and images. Start image in a sequence and end image in a sequence. Also add images that could be annotated with text. Journey should make a shorter stop on these images.

FrameAnimation Rate

Not sure this is a bug, but Im running a screen thats on 30hz, this causes my animations to be half the speed of what you probably see them in.

Create a Cover UI

The main purpose of this UI would be to prevent from loading unnecessary data, until activated.

Proposition: a container covering the whole area of the viewer, which is clickable and where clicked it initializes the viewer. Doing it that way requires finding some metaphor for activating the viewer. It's easy for platforms like YouTube or Vimeo where the play button makes sense.

What action/metaphor should we use, so we don't throw unnecessary decisions at the user? Let's discuss it here.

Create labels for issues

UIService compiler error

Locally the Typescript compiler gives the following error when running gulp ts:

TypeScript error: src/viewer/UIService.ts(59,35): Error TS2511: Cannot create an instance of the abstract class 'UI'.

This is not logged when running in CircleCI.

It is still possible to run the library in the browser.

Test in different browsers

To avoid crash.

Keyboard navigation

Keyboard navigation should handle all directions as well as panorama edges.

Naming / naming conventions for components related to UI

Continuing the discussion started by @knikel in #19. Think it is important enough to merit its own issue.

The name UI is not really capturing the whole concept of the components that live in the UI folder.

We currently have components/classes doing essentially one of the following five things in the UI folder:

GL rendering - Registers to the GLRenderer to render scenes (GLUI)
DOM rendering - Registers to the DOMRenderer to render HTML elements (SimpleNavUI)
Displaying UI - Creates and shows a complete UI (CoverUI)
Handling events - Handles DOM events and reacts on them (KeyboardUI)
Intelligent actions - Subscribes to streams and performs intelligent actions based on their combination (SimplePlayUI)

As said most of these components are not really UIs by them self. What these components do when combined together though is creating a User Experience. A GLImagePlane component, a DOMNavigation, a MouseHandler and a KeyboardHandler as well as a CacheBot and a PlayBot together creates a UX.

With this view of things the UI folder should be renamed to UX, base classes or interfaces should be named UXComponentBase or IUXComponent respectively.

For GL and DOM components the convention could be to prefix or suffix with GL or DOM. Handlers can be suffixed by Handler and bots by Bot. There could be five folders initially under the UX folder:

GL
DOM
Handler
Bot
General

Create textures without the TextureLoader in GLUI.

Avoid using the TextureLoader because it relies on the images being cached in the browser. Use the cached image element on the node class to create the texture directly using the Texture constructor.

Indicate if we are at a perspective or panorama image

With unified navigation it is hard to know if you are looking at a panorama or a perspective image. An indication of some sort is needed to let the user understand what is possible to do at the current image.

Zoom functionality

It should be possible to zoom in and out when looking at a panorama. Zoom should be a property on the state and the state should expose a method for changing the zoom. There should be max and min zoom levels.

The mouse handler should handle wheel events and zoom accordingly.

A decision needs to be taken regarding if it should be possible to move when in zoomed state.

Zoom should be reset when arriving at a perspective image.

Consider making Spatial a static library

Spatial lib is used in many places and needs to be instantiated on every place. Sine its keeping no state consider making all functions static.

Create a single set of UIs

To to launch if none are specified at initialization of Lib. Make use of this in examples so we get gl effect.

Make loading more visible

Often then sky is white. Then loading bar is not visible.

Create ObjectUI

We want to display objects located in images. There will be many sources of objects that we want to display. They do share some common attributes though.

They have a location in longitude/latitude
They contain meta information in a key/value
They may or may not be editable

I suggest the current construction of a new UI

                                            +----------------+
                                            |                |
                                            | State Context  |
                                            |                |
              +----------------+            +-------.--------+
              |                |                    .
Service X ->  | Service X Glue ..                   .
              |                | ..                 . CurrentNode
              +----------------+   ..               .
                                    ....    +-------.--------+
                                       .... |                |
                     Generic Object API   ...   ObjectUI     |
                                       ...  |                |
              +----------------+    ...     +-------.--------+
              |                | ...                .
Service Y ->  | Service Y Glue ..                   . Render in Three.js canvas
              |                |                    . (How is to be decided)
              +----------------+                    .

By using a generic Object API we are able to integrate new components by just creating a new glue component.

API 1.0.0

A suggestion and implemantaion

Uncaught error when calling Navigator.moveDir if no edge exist in specified direction

When calling and subscribing to Navigator.moveDir for a direction that is not represented among the edges for the current node the application crashes with the following error message:

Uncaught TypeError: Cannot read property 'subscribe' of null

The subscriber gets null back instead of an observable where the first item is null.

This happens everywhere where a direction is not represented, e.g. when moving in the STEP_LEFT direction for the K0vg7We6gvg4xbN0ubve1g node.

This is currently handled in the keyboard handler by checking if the direction actually exists before calling Navigator.moveDir.

Try virtual DOM implementation for the UI tree

Clear functionality in DOMRenderer

It should be possible for a DOMUI to clear its hash and therefore its content in the virtual DOM with an method on the DOMRenderer similar to the functionality in the GLRenderer:

mapillary-js/src/render/GLRenderer.ts

Lines 308 to 310 in ea9e928

 public clear(name: string): void { 

 return this._clear$.onNext(name); 

 }

mapillary-js/src/render/GLRenderer.ts

Lines 132 to 140 in ea9e928

 this._clear$ 

 .map<IGLRenderHashesOperation>((name: string) => { 

 return (hashes: IGLRenderHashes): IGLRenderHashes => { 

 delete hashes[name]; 

 return hashes; 

 }; 

 }) 

 .subscribe(this._renderOperation$);

DOMUIs should clear their hash (and release potential memory) on deactivation.

Adjust to different aspect ratios within GL handlers and renderer

Right now the GLUI is hardcoded to a 4:3 aspect ratio. Enforcing the container to a 4:3 ratio can by done in CSS. This is not what we want though.

The GLRenderer and the WebGL handlers should be able to handle all aspect ratios, either through letterboxing or through filling the screen in the optimal way depending on image type, image orientation and current aspect ratio.

Preferably the letterbox/fill behaviour should be configurable.

Only uglify in production

For easier debug only uglify sourcecode in production

Find way to illustrate next in sequence in DirectionsUI

Example pvuThYMsqOz1syYCXDaiHQ , going real linked images will only take you that far.

However if there is a good amount of alternatives we might not wanna show sequence arrows.

Perspective navigation UI

Create a navigation UI with step arrows pointing in the correct world direction with respect to the current camera.

Reset graph when lat lon of current node changes a lot

When the latitude or longitude of the node that is to be set on state changes a lot the graph should be reset and the reference lat lon updated according to the new node.

The trajectory of the state needs to be handled, probably reset.

Panorama support

Render panorama images on spheres in GLUI.
Look around functionality in State.
Mouse drag handler using state API for looking around.

Use versioned URLs for gist examples and README

Fix a CDN urls for various releases. Update README.md to reflect that change

What should be a default set of enabled UI?

It would be great if we could start deciding what set of UIs should be enabled by default in mapillary-js. Below is my take on what's essential for the initial version. Bear in mind that some of these could be disabled, with an option flag when initializing the viewer.

Current list of UI/features we should take into consideration:

AttributionUI - Displays attribution for the user, links to mapillary.com
CoverUI - Downloads a 320px photo, sets it as a background cover above all the other UI elements in the viewer container. Displays a button which will activate other UIs and will start fetching data in the future (doesn't work yet).
DetectionsUI - Makes an API call to /or endpoint for the given photo, displays rectangles with detected traffic signs on top of the viewer.
EventUI - Emits various events, to which mapillary-js user can hook into
GlUI - The heart of mapillary-js 😉
KeyboardUI - Allows navigation using the keyboard
LoadingUI - Display loading progress as a 2px bar in the top of the viewer.
SimpleCacheUI - Caches the fetched data in case the user returns to the photo previously visited
SimpleNavUI - Displays direction arrows in various corners/at various edges of the viewer.
SimplePlayUI - Allows to play/stop playing of sequences
SimpleUI - Canvas based viewer

Proposition: I think, we should have a set of sane defaults for most uses. Let's start with the most important UIs: GlUI and SimpleUI. I would say that we should also have GlUI enabled by default, and SimpleUI should be its' fallback in case the system where mapillary-js is displayed doesn't support WebGL. Next, DirectionsUI should always be displayed, but we should provide means to toggle it. For the SimpleUI, when no WebgGL has been detected, we fall back to SimpleNavUI, since this viewer won't display panoramas. CoverUI should be also enabled by default, since it prevents loading unnecessary data and handles the scenario when multiple viewers are within the one view. AttributionUI is essential too as we want to give credit to the community (username links to the profile) and there is also a link to the photo on mapillary.com. We should display loading progress by default through LoadingUI. SimpleCacheUI should be enabled by default too, so we don't make unnecessary requests. EventUI is a no brainer, we want everyone to be able to hook into the events without doing any extra work on their side.

Proposed essential UIs/features: GlUI, CoverUI, LoadingUI, Directions, SimpleCacheUI - SimpleUI would be used only after internal feature check.

Regarding optional UIs… KeyboardUI is a tricky one, since the library consumer might have event handlers attached to keys already. I would say this should be enabled when specified. Last, SimplePlayUI, this is a really cool feature, but I'm totally unsure whether it should be enabled by default? mly.Viewer.play() is an appealing option, though.

WDYT?

Motionless transition logic

Create functionality in state regarding motionless transitions and handle when showing image planes in GLUI.

Better sequences in integrations examples and README + vector tiles in examples

Lets find some better sequences and also update js to the latest version

Update README with better panorama and perspective photo keys
Update examples with better panorama and perspective photo keys
Add vector tiles to the examples

Avoid setting the same node twice on state when loading

If the navigator is loading it is possible to request the same node again causing immediate motionless transitions to the node once cached.

Create PlayUI

Add button that will be either stop symbol or plat symbol.

Handle non merged node in mesh caching

Meshes does not exist for non merged nodes so the node should handle this when caching image and mesh.

Cleanup Documentation

Clean up documentation for 1.0 release my suggestion is.

1. Remove all classes except Viewer
2. Add the IViewerOptions Struct

	public clear(name: string): void {
	return this._clear$.onNext(name);
	}

	this._clear$
	.map<IGLRenderHashesOperation>((name: string) => {
	return (hashes: IGLRenderHashes): IGLRenderHashes => {
	delete hashes[name];

	return hashes;
	};
	})
	.subscribe(this._renderOperation$);