bkazic / mobis-traffic-prediction-node

MobiS traffic prediction prototype with node.

[MobiS] Traffic Prediction Service

MobiS traffic prediction service is open-source data analytics application for processing real-time traffic streams providing online short-term traffic forecasts. Services provides forecasts for different traffic parameters (speed, flow, occupancy, traffic-status), for various forecasting horizons (from 1h, to 24h) into the future.

Service provides:

• REST APIs - used for pusing raw data (loop counters or bluetooth sensors) and retrieving processed data
• Visualisation - used for visual investigation of current traffic and predicted traffic
• Traffic Expert - is aplication build on top of this services, used by traffic experts in Thessaloniki, Greece

Keywords: short term traffic prediction, traffic analysis, data analytics, machine learning

Install

This repository contains 3 different services with instructions:

• TrafficPrediction - main module that accepts measurement records nad performs advanced analytics.
• RealTimeImputor - fetches real time data from opendata.si and pushes it to TrafficPrediction module.
• Imputor - pushes saved batch data to TrafficPrediction module.

Prerequisites

Service is designed on top of QMiner - analytical platform for large-scale real-time streams, containing structured and unstructured data. Development of MobisTrafficPrediction service also contributed to development of QMiner platform.

Service is designed as Node.js web application, and therfore requires:

• node.js v0.12 x64 or x86,
• npm v2.11 or higher.

Acknowledgments

Service is developed by AILab at Jozef Stefan Institute for the purpose of European FP7 project MobiS.

The authors would like to acknowledge funding from the European Union Seventh Framework Programme, under Grant Agreements 318452 (MobiS), and European Union's Horizon 2020 research and innovation programme under grant agreement No 636160-2 (Optimum).

mobis-traffic-prediction-node's Issues

Do something about traffic-status 6

Traffic status 6 means some kind of error and it usually happens during the night, where it jumps from status 0 to status 6 instantly. Since we are predicting traffic-status as a regression problem (numerical values) and not classification problem (categories), this can be a problem.

Deploy to aidemo.ijs.si

• Deploy the mater service to aidemo.ijs.si.
• Service should start automatically when computer starts.
• Not sure if it should start as a backed service or as a process in cmd. Eather way, it should use something like forever, so that it automatically restarts if it crashes.

Base is closed after backup and reopen

Server still holds old instance of base that was closed with backup. New instance of base has to be somehow known to server.

Retrain the models

After model changes, retrain the models on all historical data.
Create new release 3.0.0 with new models and data.

Service should restart immediately if it crashes

Server should restart immediately if it crashes.
You can try to use node module Forever, or something similar.

Use traffic.ijs.si

Redirect traffic to http://traffic.ijs.si/

Fix all redirections.

Problem:

• http://traffic.ijs.si/ - redirected to http://mustang.ijs.si:8011/sensors.html - Doesn't work!
• http://traffic.ijs.si/counters - redirected to http://aidemo.ijs.si:9569/ - Doesn't work!
• http://traffic.ijs.si/counters/ - redirected to http://aidemo.ijs.si:9569/ - Works!

Repository clean up

• Update README.md
• Delete unused branches
• Sync 'master' with 'development'
• Check if file SaveLoadTest.js is needed
• ...

TypeError: Cannot read property 'admin' of undefined

TypeError: Cannot read property 'admin' of undefined

This happens at HTTP call: /get-store-recs/:store
Response from the server is: 500 (Internal Server Error)

Opening saved state is not working properly

Records between latest data in (saved) database, and new data after reopening (loading) saved models, are not resampled correctly.

Compress the latest backup

Create .zip or .rar from the latest ./backup folder. Just in case something goes wrong during the opening from backup - this would corrupt backup and we would loose all the data.

[RealTimeImputor] RangeError: Invalid time value

When fetching data with RealTimeImputor, this error occurs:

RangeError: Invalid time value at Date.toISOString (native)

Evaluation visualization doesn't work any more

Check why evaluation visualization doesn't work any more.

Organize logging better

Currently there is too much logging to the console when NODE_ENV=production.

Change default `npm start` to run the service in production mode

Change default npm start in package.js to start in NODE_ENV=production.
Also you can try with default open mode. If the database is corrupted or missing, log to console to use cleanCreate.

Do not calculate predictions if there is no input data

Do not calculate predictions if there is no input data, since the predictions are poor when there is a long period of missing data.

Implement Save and Load functionality

This will enable also creating backups and loading from backups.

Import visualization

Import visualization from https://github.com/JozefStefanInstitute/mobis/blob/predictionsVisualization to this repo.

Pre-process speed 0 during the night

Pre-process speed 0 that occurs during the night.
This was already solved in old versions.

Update project to latest QMiner release

Update to QMiner v3.1.0