project-osrm / osrm-text-instructions

Text instructions from OSRM route responses

OSRM Text Instructions

OSRM Text Instructions is a Node.js library that transforms route data generated by OSRM into localized turn instructions to be displayed visually or read aloud by a text-to-speech engine. OSRM Text Instructions is the basis of guidance instructions in osrm-frontend, the Mapbox Directions API, and the Mapbox Navigation SDK.

• Global: Text instructions are available in over a dozen languages via Transifex. Abbreviations and advanced grammatical transformations are available in some languages.
• Customizable: Flexible options allow you to format and tweak the results to your liking.
• Cross-platform: A data-driven approach facilitates implementations in other programming languages. OSRM Text Instructions is also available in Swift and Objective-C (for iOS, macOS, tvOS, and watchOS) and in Java (for Android and Java SE).
• Well-tested: A data-driven test suite ensures compatibility across languages and platforms.

Usage

var version = 'v5';
var osrmTextInstructions = require('osrm-text-instructions')(version);

response.legs.forEach(function(leg) {
  leg.steps.forEach(function(step) {
    instruction = osrmTextInstructions.compile('en', step, options)
  });
});

If you are unsure if the user's locale is supported by osrm-text-inustrctions, use @mapbox/locale-utils for finding the best fitting language.

Parameters require('osrm-text-instructions')(version)

parameter	required?	values	description
version	required	v5	Major OSRM version

Parameters compile(language, step, options)

parameter	required?	values	description
language	required	en de zh-Hans fr nl ru and more	Compiling instructions for the selected language code.
step	required	OSRM route step object	The RouteStep as it comes out of OSRM
options	optional	Object	See below

Options

key	type	description
legCount	integer	Number of legs in the route
legIndex	integer	Zero-based index of the leg containing the step; together with legCount, this option determines which waypoint the user has arrived at
formatToken	function	Function that formats the given token value after grammaticalization and capitalization but before the value is inserted into the instruction string; useful for wrapping tokens in markup
waypointName	string	Optional custom name for the leg's destination, replaces "your {nth} destination"

formatToken takes two parameters:

• token: A string that indicates the kind of token, such as way_name or direction
• value: A grammatical string for this token, capitalized if the token appears at the beginning of the instruction

and returns a string.

Architecture

• index.js contains the main transformation logic in JavaScript.
• languages/ contains the localization files, including raw format strings, abbreviation files, and grammar rules.
• languages.js loads the localizations and contains some localization-related helper functions.
• test/ contains data-driven integration tests and test fixtures for all supported languages.

Contributing

We welcome feedback, code contributions, and translations! Please see CONTRIBUTING.md for details.

osrm-text-instructions's Issues

Error running npm run transifex

As of #107, npm run transifex fails with the following error:

> [email protected] transifex /Users/mxn/hub/osrm-text-instructions
> node scripts/transifex.js

/Users/mxn/hub/osrm-text-instructions/scripts/transifex.js:26
Object.keys(languages.codes).forEach((code) => {
       ^

TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)
    at Object.<anonymous> (/Users/mxn/hub/osrm-text-instructions/scripts/transifex.js:26:8)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
    at Module.runMain (module.js:605:10)
    at run (bootstrap_node.js:423:7)
    at startup (bootstrap_node.js:147:9)

/cc @patjouk @freenerd

Handling mode changes

Currently modes are incorrectly modeled as turn types. This is not how OSRM returns these.

I believe travel modes are fairly static in OSRM, so these could become part of the instructions.json

• https://github.com/Project-OSRM/osrm-backend/blob/c306a5985476b4008885296b1abd02912097c161/include/extractor/travel_mode.hpp#L42-L54
• https://github.com/Project-OSRM/osrm-backend/blob/c306a5985476b4008885296b1abd02912097c161/profiles/bicycle.lua#L383-L384

What I'm not clear is if we can avoid having access to the previous step to calculate if something changed. Might need to have previous and current 😢

One Step at a Time vs Combined Steps

Currently the design is to handle one RouteStep object at a time. I'm questioning this approach, and I'm wondering if the logic has to be more advanced.

Think about this example: you get a step for left, and then a step for right immediately after. There is neither much time nor much distance in between these steps. Do you really want to announce:

Make a left turn onto X
Make a right turn onto Y

or would it be more useful to combine those into a single text instruction

Make a left immediately followed by a right turn

(note: the example is simple and the same can happen with all instructions)

On the backend side we're already trying to reduce subsequent short steps when possible, e.g. Staggered Intersections: Project-OSRM/osrm-backend#2738

But there are still tons of scenarios where a combined text instruction for multiple steps would make sense. In the backend we can't combine all instructions after all, since we're working on instruction/modifier primitives.

How do you want to handle these with this project? Is this out of scope? And if so, how would a client using this project combine text on its own?

Integrating phonemes

I know that Phonemes are a weird topic:

• not much used in OSM
• few open source implementations that process them well as audio

Still, a hot topic for doing really good voice guidance. How would we integrate them? Do we need phonemes for instructions as well, or only for street names?

• Phonemes for instructions They could become their own language to integrate, thus we'd have spoken language and phoneme scheme combinations
• Phonemes for streets These comes from OSRM and is already implemented as the pronunciation property for street names. They'd just be put into the instructions string then, so this would be an easy options to the instruction logic if it should look for pronunciations or not.

Merge slightly/sharply left/right should be merge left/right

There are instructions like “Merge slightly left towards {destination}”. Adverbs like “slightly and sharply” should be removed from merge instructions.

One does not merge “slightly” or “sharply” to the left or right. When you say “merge left”, “left” indicates which side the other traffic is coming from, not which way you’ll turn, so there’s no distinction between slight, normal, and sharp.

/cc @freenerd @bsudekum

"Stay on" when continuing on the same road

Along the same lines as #30, it would be nice if we could say "Keep left to stay on Main St" if Main St. is the street both before and after the fork in the road.

/cc @freenerd @bsudekum @mayagao

Improve lane instruction

Currently:

Keep in the middle

Should be:

Use the middle lane

/cc @pveugen

Chinese translation

Would be great to test how well instructions and our tokenization approach work in chinese.

/cc @YunjieLi

Ruby port

I didn't know how to contact you guys, so I'm creating this issue, I hope that's not a problem.

Just so you guys know, I know it's very soon but I've ported this lib almost verbatin to a ruby gem: https://github.com/zavan/osrm_text_instructions

I plan to keep it up to date with this lib.

Thanks for the great work.

Spanish translation

I have some instructions translated if it helps: http://www.filedropper.com/spanishtranslation_1 It would be very nice if you could implement it! Thanks and great work!

Automatically generate fixtures

In https://github.com/Project-OSRM/osrm-text-instructions/blob/master/scripts/generate_fixtures.js we already have some automated fixture generation code, that can be extended e.g. like #32.

The code is ugly, but actually has been very helpful. I think there is high value in generating all fixtures automagically, so we make sure they will never be stale.

Sometimes emit lane information in instructions

Currently we only emit lane information during type: use lane.

We should do that in other cases as well. Most cases we don't need to do that (e.g. on simple forks or turns) but identify cases like "stay right to turn left" where it will make sense.

How to deal with Arrivals?

Currently we have one step in OSRM for the arrival. We are emitting one instruction for it:

{
    "step": {
        "maneuver": {
            "modifier": "left",
            "type": "arrive"
        },
        "name": "Street Name"
    },
    "instruction": "You have arrived at your destination, on the left"
}

To deal correctly with arrivals, we would actually need TWO instructions though, one after the last instruction before arrival, one on arrival:

• In 200 meters, you will arrive at your destination, on the left
• You have arrived at your destination, on the left

There are several solutions here:

1. The client could deal with this exclusively (given that they also already implement distances). The text instruction would need to be in the present then:

• In 200 meters, you arrive at your destination, on the left
• You arrive at your destination, on the left

2. We make osrm-text-instructions alert level aware

• Needs a new ticket, i'm still undecided if and how we should do this

3. We add a new arrivals type in osrm-backend

• This sounds like it would be way too much effort with something that is actually very specific to the layers above.

For now I'd go with option 1.

/cc @MoKob @daniel-j-h @TheMarex @danpat @1ec5 @willwhite @pveugen

Can we reduce special logic in index.js even more?

The smaller this logic is, the more extensible this library will be. We should try to see if we can reduce this logic even further.

/cc @freenerd

Use destinations

We should include destinations in the text instructions.

It should say Take the ramp toward Baltimore.

We might be able to tack this onto the {way_name}, but then toward would not be localizable.

/cc @bsudekum

Never allow english strings outside of instructions.json

Can we lint for this?

/cc @freenerd

Roundabouts/Rotaries should include exit numbers and street name

First, roundabouts should have the same handling as rotaries.

Then, currently roundabouts only cover the first case, but instead should cover all. Let's stay in line with #8 and have all of the string in the instructions.json. Maybe this is the point where we'd need a better templateing language, that can support nesting of attributes, so that we could have a string like this:

Enter {rotary_name} [=nameandexit and take the {exit} exit onto {way_name}] [=name and exit onto {way_name}] [=exit and take the {exit} exit]

    case 'rotary':
        instruction = instruction.replace('{rotary_name}', step.rotary_name || 'the rotary');
        if (step.name && m.exit) {
            instruction = instruction + ' and take the {exit} exit onto {way_name}';
        } else if (step.name) {
            instruction = instruction + ' and exit onto {way_name}';
        } else if (m.exit) {
            instruction = instruction + ' and take the {exit} exit';
        }
        break;

Use ref and name together

Currently step.ref is ignored, but shouldn't be.

Allow better language file handling

Currently all language files are loaded when osrm-text-instructions is initialized, but only one is used then.

We should instead:

• Allow to control which languages are actually loaded
• Allow switching languages without reloading language files

Callback for token replacement values in compile

#47 feels brittle, since it depends on the format of the instructions.json and therefore couples code tightly.

Instead, @1ec5 proposed to introduce a hook just before the final token replacement. That would be a more-stable API, while allowing even more functionality.

handle waypoints

ref

osrm-text-instructions/index.js

Line 38 in f1d966c

// TODO, add wayPoint argument

Access previous/next step during compile

Currently, the compile(step) function only has the current step as input. There are cases that we can only solve by allowing access to the previous/next step:

• Mode changes #18
• "Stay on ..." instruction #41
• Combined instructions #15

Remove .js from repo name

I think osrm-text-instructions.js is misleading, since we ideally want this to be used by non-js projects as well.

@MoKob can you rename this repo name to osrm-text-instructions?

@bsudekum headsup for change of git remote.

Roundabout text instruction does not replace name correctly

http://map.project-osrm.org/?z=12&center=50.517902%2C4.766350&loc=50.504256%2C4.792099&loc=50.565794%2C4.827461&hl=en&alt=0

Enter the roundabout and take the second exit onto {way_name}

Enable travis

@MoKob can you enable Travis on this repo? There is already a .travis.yml included.

Ordinalize to 25

Once #7 is in we should count higher than 10 right now. 25 seems like a good number, since that's the limit of waypoints in the Mapbox Directions API

Use numeric ordinals

Let's use numeric ordinals, since they are easier to read and what we can use native translation functions on iOS. Assumption is that Text-To-Speech Engines can deal well with these.

Distance-based instructions

Turn-by-turn navigation applications need to say things like “In {distance}, {instruction}” (as the user approaches a maneuver) and “Continue on {name} for {distance}” (immediately after the user completes a maneuver). It’s easy enough for the application to provide these strings itself, but then it may not be localized as broadly as the strings in this project, and {name} won’t reflect osrm-text-instructions smarts like #77.

/cc @freenerd @bsudekum

Make instructions.json format localizable

For the instructions strings file, we should use a format that standard localization tools can work with. What we have now is more or less the JSON key-value format supported by Transifex, but with an informal mini-language around placeholders and optional clauses. This format has some downsides for localization:

1. No way to provide comments to localizers
   • Switching to Chrome JSON format would address this issue.
2. No way to vary capitalization or word order when a clause is omitted
   • We should get rid of all [optional] clauses and enumerate all the possibilities. String concatenation always works against localizability.
3. No way to vary surrounding words based on a placeholder’s number or grammatical gender
   • We could build this ourselves by allowing any string to be replaced by an object mapping number patterns (ONE, TWO, MANY) to message variants, but localizers would be unable to add their own variants using a tool like Transifex.

Primarily for (3) and so that other clients don’t have to build their own custom parsers, we should consider switching to one of the other Transifex formats that happens to support these features. As an iOS developer, I’m most familiar with .strings and .stringsdict, but gettext and XLIFF are solid formats that support all the features we need for localization while being compatible with Node, Android, and iOS development.

Depends on #6 and #5.

/cc @bsudekum @MoKob @freenerd @zugaldia

Transifex script should use tx

transifex.js should use the tx command line client, which provides a better authentication mechanism and options for some of the things we’re doing manually in that script, like mapping language codes. For inspiration, the mapbox-gl-native project uses tx with a simple configuration file.

/cc @freenerd @frederoni

Can't extend {way_name} with an option on Transifex

I prepared new feature for OSRM Text Instructions - grammatical cases support for way names. This is quite important for many languages (Russian, Ukrainian, Polish, Bulgarian, Finnish, Estonian and many others) - otherwise translated strings with inserted way names (in nominative case as in OSM) look grammatically incorrect in most cases. Moreover, I don't know any other navigation system that could do this properly (except my humble attempt to improve Garmin TTS voices 😉). I would be proud to make OSRM Text Instructions the first system with grammatical case support but I can't 😞

The main idea of this future feature is in extending {way_name} and {rotary_name} variables in target language's strings with explicit grammatical case label - {way_name:accusative} for example - and processing way names with block of regular expressions grouped by this grammatical case.

But Trasifex rejects my attempts to change "Turn left onto {way_name}", in Russian "Поверните налево на {way_name}" to "Поверните налево на {way_name:accusative}" with "Missing {way_name} variable error" 😥 Other variants ({way_name}{accusative} for example) also don't work due to difference with variables from original English string.

My experience with TTS voices reveals that determining required grammatical case for part of whole text is not the simple task and could be easily done only for known and expected words combinations. Actually this could be implemented too but extends already big regular expressions file (already ~900 expressions for Russian street names only) some more.

I don't like the extending all translation strings with proposed grammatical case option - this has no sense for them. I don't want to extend English strings only with this option - this invalidates all existing translations. I would like to update Russian (currently) translation only. Maybe some Transifex checks could be suppressed/updated to allow non-exact match of variables?

@1ec5, @freenerd, @TheMarex, need your advice please.

Handle type notification

We should handle type: notification. This is tightly related to handling mode changes #18

Dealing with change in translations

We will soon run into problems with translations getting out of sync with expected formats, since we can't get the translation in fast enough. Candidates for such change are #41 and #61.

Our gold standard is the english translation, all others follow. Just now, I think don't want to commit to guarantees regarding translations. We should do that eventually (e.g. every major release will include consistent translations).

For now we should establish fallbacks what happens when a translation is not available. I see two options:

1. Fallback to english translation
2. Fallback to not announcing the step as expected, but treat it as a turn instead

I guess option 2. might involve some tricky edge case handling around detecting that a translation is not there. It may also result in bad guidance. Option 1. on the other hand will certainly alienate some users. On the other hand, we can check for this case very easily during running tests and detect when a feature is missing. So I'm leaning towards option 1.

Opinions?

Use most common/popular ref and destination

Ways can have several ref and destinations. These are usually ; delimited. Currently we are only always using the first one. Instead, we should use the most common or popular one.

As an example, imagine a cross-country road that has the following refs on the way:

• I-80
• I-80;I-81
• I-80;I-81;DETOUR-20
• I-80
• I-90;I-80
• I-90;I-80
• I-80

It would be great if the user would always only get I-80 as the ref.

We could implement this via a pre-processing step where we create a priority list (based on occurence, maybe mixed with length of staying on segments) and when deciding for a ref within a way, we should use the one with highest priority.

Allow for translations

We could probably move instructions.json to a language folder, then have different json files for each language.

Use lower-case language codes everywhere

We currently have zh-Hans which may result in confusion when people use zh-hans instead. Let's eliminate this source of confusion:

• Make sure all language codes in

  osrm-text-instructions/languages.js

  Lines 16 to 27 in 145f4f3

  	var instructions = {
  	'de': instructionsDe,
  	'en': instructionsEn,
  	'es': instructionsEs,
  	'fr': instructionsFr,
  	'id': instructionsId,
  	'nl': instructionsNl,
  	'ru': instructionsRu,
  	'sv': instructionsSv,
  	'vi': instructionsVi,
  	'zh-Hans': instructionsZhHans
  	};

  are lower-case
  • change zh-Hans to zh-hans
  • write a test that fails if a non-lowercase character is added
• Make compile(language) accept any-cased language codes by lower-casing the input code (and have a test for this)

Better U-turn instructions

Although there are some combinations that I doubt would happen, we should still cover all cases with understandable instructions. example https://github.com/Project-OSRM/osrm-text-instructions/blob/master/test/fixtures/v5/fork/uturn.json

{
    "step": {
        "maneuver": {
            "modifier": "uturn",
            "type": "fork"
        },
        "name": "Street Name"
    },
    "instruction": "Keep uturn at the fork onto Street Name"
}

Add eslint

The js code would benefit from some linting.

Create fixtures for all translated languages

For working with the english language instructions, looking at the actual output instruction strings is very helpful. Maybe we should do the same thing for all language, especially since we may end up having to implement some language-specific logic.

The requirement for this is having #54 working for the complete fixture suite.

Swift port

Documenting that a swift port will happen:

• Transform instruction files to .plist
• Execute integration test suite
• Build and package

/cc @1ec5 @bsudekum

Pre-compute information before compile

We might want to introduce a pre-compute step before compile that extracts information from the complete OSRM route response. This would allow handling of the following cases:

• Use common/popular ref/destination #30
• Handle waypoints #7

Handle type:continue modifier:uturn better

See Project-OSRM/osrm-backend#3082

That should become Make a uturn ...

Scope language files by osrm version

Right now we have the osrm version as part of the language files. That's like not going to be useful in the future, since we'd blow up the language files a lot with a new version.

Instead we should create own files per osrm version and then switch between these. We already do that for test fixtures. Likely we'd want to do the same for the core instruction logic as well.

Consumers can still support the unlikely case of supporting several OSRM versions at the same time by loading the objects several times. This is only a question of repo file structure and how much data to load into memory.

Put translations into Transifex

We should put the language files into Transifex. Let's wait with this a bit though, until we are sure the current format and strings are stable though.

Sometimes use only name or ref, not both

There are cases where it would be better to ignore the name and only use the ref

http://map.project-osrm.org/?z=15&center=38.868532%2C-77.056947&loc=38.877670%2C-77.040553&loc=38.868248%2C-77.048450&hl=en&alt=0

There are cases where it would be better to ignore the ref and only use the name

http://map.project-osrm.org/?z=16&center=52.518304%2C13.445184&loc=52.517331%2C13.453681&loc=52.522501%2C13.450441&hl=en&alt=0

Doing this is a hard problem, since it requires local knowledge. I do not know how we'd solve this.

Instructions containing pedestrian tokens

using the foot profile many names of steps are tokens like: {highway:pedestrian}, {highway:service} or {highway:footway}

the compiled instructions are containing these tokens:
Turn right onto {highway:footway}

Omit redundant lane information

"Keep left to turn left" is redundant. We should only indicate lanes when they contradict the user's intuition, like for Melbourne-style left-lane right turns.

/cc @freenerd @bsudekum

Add destination information to ramp types

Where a destination is provided, it should be used in place of name for ramps.

/cc @freenerd

Remove number-to-words js module

We are using number-to-words. I think we'd want to remove the dependency completely:

• we need to own all translation strings ourselves
• the module won't be available in other programming environments than javascript

Investigate why ref is not used

http://map.project-osrm.org/?z=15&center=32.363035%2C-86.321898&loc=32.367819%2C-86.314688&loc=32.360751%2C-86.321812&hl=en&alt=0

Has only ref and should display that