dvdsk / dataserver Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 0.0 1.3 MB

Rust 84.44% HTML 6.61% CSS 2.76% JavaScript 4.90% Dockerfile 0.28% Handlebars 0.98% Shell 0.03%

dataserver's People

Contributors

Stargazers

dataserver's Issues

Dynamic time ranges

needs:

javascript and html changes to facilitate choosing an interval
mechanism to send multiple websocket packages for init data (js and rust changes)
averaging over existing data to lower data usage

slice index starts at 3 but ends at 2

Error in compression lib:

stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:476
   5: std::panicking::continue_panic_fmt
             at libstd/panicking.rs:390
   6: rust_begin_unwind
             at libstd/panicking.rs:325
   7: core::panicking::panic_fmt
             at libcore/panicking.rs:77
   8: core::slice::slice_index_order_fail
             at libcore/slice/mod.rs:2340
   9: <core::ops::range::Range<usize> as core::slice::SliceIndex<[T]>>::index_mut
             at libcore/slice/mod.rs:2509
  10: core::slice::<impl core::ops::index::IndexMut<I> for [T]>::index_mut
             at libcore/slice/mod.rs:2327
  11: dataserver::httpserver::timeseries_interface::compression::encode
             at src/httpserver/timeseries_interface/compression.rs:51
  12: dataserver::send_test_data
             at src/main.rs:112
  13: dataserver::main
             at src/main.rs:216
  14: std::rt::lang_start::{{closure}}
             at libstd/rt.rs:74
  15: std::panicking::try::do_call
             at libstd/rt.rs:59
             at libstd/panicking.rs:310
  16: __rust_maybe_catch_panic
             at libpanic_unwind/lib.rs:102
  17: std::rt::lang_start_internal
             at libstd/panicking.rs:289
             at libstd/panic.rs:392
             at libstd/rt.rs:58
  18: std::rt::lang_start
             at libstd/rt.rs:74
  19: main
  20: __libc_start_main
  21: _start

Encode plot request info in url

This would facilitate easier sharing of plots and setting a certain "page" as bookmark. For example temperature in the last day.

For possible format check industry standards

High virt mem usage

might be related to: #9

refactor botuserinfo and userinfo

botuserinfo => botuser
userinfo => user

username => name
timeseries_with_access => access or maybe data_access

cant send large amounts of data without increasing ram usage

related to #13, need infastructure to send multiple init_data websocket packages

Certificate update no longer working

Probably related to move to https://github.com/dskleingeld/cert_manager which has numerous updates atm

Decimal numbers are not recognised in bot plot command

example input: /plot 3:1 1.5d

gives output:

One of the arguments could not be converted to a number
use: /plot <plotable_id> <number><s|m|h|d|w|monthes|years>

Crash on adding dataset access after adding telegram id

steps to reproduce:

create new user
add telegram id
save
[optional] try to use the bot (will show that the account has access)
add dataset access (at least one field)
save
[optional] try to use the bot (will show that the account has no access)
change telegram id for user
save

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: UserDatabaseError(UserNotInDb)', src/libcore/result.rs:1165:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:77
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:61
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1028
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1412
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:65
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:50
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:188
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:205
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:464
  11: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:373
  12: rust_begin_unwind
             at src/libstd/panicking.rs:302
  13: core::panicking::panic_fmt
             at src/libcore/panicking.rs:139
  14: core::result::unwrap_failed
             at src/libcore/result.rs:1165
  15: core::result::Result<T,E>::unwrap
             at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libcore/result.rs:933
  16: dataserver::menu::user::menu
             at src/menu/user.rs:61
  17: dataserver::menu::command_line_interface
             at src/menu/mod.rs:31
  18: dataserver::main
             at src/main.rs:94
  19: std::rt::lang_start::{{closure}}
             at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/rt.rs:61
  20: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:48
  21: std::panicking::try::do_call
             at src/libstd/panicking.rs:287
  22: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:78
  23: std::panicking::try
             at src/libstd/panicking.rs:265
  24: std::panic::catch_unwind
             at src/libstd/panic.rs:396
  25: std::rt::lang_start_internal
             at src/libstd/rt.rs:47
  26: std::rt::lang_start
             at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/libstd/rt.rs:61
  27: main
  28: __libc_start_main

Gaps and incorrect values in recieved data

see the following example plot:

u32 values fail to encode/decode

Seems to be caused by an uncorrect value for stop_byte

Actix actor system no longer working

This prevents the alarms from working and causes the live web plots to stop functioning

refactor encode en decode function

has an if statement that should be possible to remove, is hard to maintain

refine error management for multiple field sensors

look at hash-map keys, these don't work for multi field
edit error messages to hide sensor fields that are non relevant

replace raw rust variables with serde name statement

#[serde(rename = "name")]
see: https://serde.rs/field-attrs.html#rename

plot command, number of minutes:hours:etc is not parsed correctly

fails at:
float numbers (for example 1.2h)

Systemd integration

use a lib like: https://docs.rs/sd-notify/0.1.1/sd_notify/ or https://docs.rs/libsystemd/0.1.0/libsystemd/

to signal when errors occur or the service has started

if no data lies within requested time range no metadata is send

see line warn!("could not setup read"); and warn!("no data within given window"); in data_router_ws_client.rs and the consequences of those branches

spec field type bitlength does not use the full range available

No scaling is introduced, for example:

  - BitLength:
      name: Temperature
      min_value: -20.0
      max_value: 50.0
      numb_of_bits: 10

results in:

  - id: 0
    name: Temperature
    offset: 0
    length: 10
    decode_scale: 1.0
    decode_add: -20.0

long login

Logging in takes 600 millisec, from start op post call till end

No data send on plot command

updates do kinda work.
seems to go wrong in client side allocation of array, number of lines is non integer.

panic on plot while running on rpi

log/trace

[12:11][dataserver::httpserver][TRACE] checking login       
[12:11][dataserver::httpserver][INFO] user logged in    returning file: "web/plot.js"                            
[12:11][dataserver::httpserver][TRACE] websocket connected
[12:11][dataserver::httpserver::websocket_client_handler][TRACE] sending data to client                            
[12:11][minimal_timeseries][DEBUG] header data {1545869954:0}                                                     
[12:11][minimal_timeseries][WARN] start TS earlier then start of data -> start_byte = 0                  
[12:11][minimal_timeseries][DEBUG] header data {1545869954: 0} 
[12:11][minimal_timeseries][DEBUG] start time: 2017-08-14 11:11:50.593609283 UTC, 1502709110;end_time: 2018-12-27 11:11:50.593635689 UTC, 1545909110 
[12:11][minimal_timeseries][DEBUG] start_byte: 1805
[12:11][minimal_timeseries][DEBUG] current_timestamp: 154586
9954, next_timestamp: 9223372036854775806, next_timestamp_pos: 14137, start_byte: 0, stop_byte:14130                 
memory allocation of 500000000 bytes failedAborted

X axis confusing

See the plot below, the x-axis is the minute part of the hour. however this is not intuitive if you look at it.

Suggestions for improvement:
add hours thus mm:ss -> hh:mm:ss
or switch to numb of minutes ago

Certificate file names confusing

unconfirmed but:

signed cert = keys/cert/cert
intermediate = keys/intermediate.cert
private key = keys/cert.key

Let datarouter store new data

This would allow us to directly return the nodes http requests, saving power by allowing sleep to resume earlier.

Server stop recieving data

Sometimes resumes after plot request from web client.

Deleting dataset results in panics

When a dataset is deleted on disk it still persists in the user database. The program assumes all datasets in a users metadata exist.

possible resolutions:
-check on user load if user info still valid
-add a delete function from server interface

Add Resolution and range based spec

instead of

SigDigits:
name: temperature
min_value: -20.0
max_value: 60.0
number_of_digits: 4

we could have

ResolutionAndRange:
name: temperature
min_value: -20.0
max_value: 60.0
resolution: 4

dvdsk / dataserver Goto Github PK

dataserver's People

Contributors

Stargazers

dataserver's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs