GithubHelp home page GithubHelp logo

apache / streampipes Goto Github PK

View Code? Open in Web Editor NEW
566.0 27.0 173.0 314.47 MB

Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.

Home Page: https://streampipes.apache.org

License: Apache License 2.0

Java 58.65% Shell 0.61% HTML 8.91% Dockerfile 0.17% JavaScript 0.14% TypeScript 21.41% CSS 0.01% Python 2.41% SCSS 3.16% Makefile 0.02% Jupyter Notebook 3.88% Go 0.64%
iot analytics edge stream-processing iiot self-service hacktoberfest

streampipes's Issues

Links are out of date

  • Links at "User Preferences" -> "Info" should be updated
  • Link "User Preferences" -> "Documentation" is not working

Imported from Jira STREAMPIPES-84. Original Jira may contain additional context.
Reported by: tex.

Comparison to StreamSets

Hey, so I stumbled across this project and looked interesting, so digging around I see it's just a graphical editor on top of a bunch of Java SDKs, so I am really curious what the differences might be and why someone may chose your product over another?

Dashboard Widget without Parameters

It should not be possible to create a new Dashboard widget unless all required paraments/configurations are set. If a widget is created without all the required paraments, the dashboard will 'crash'.

Imported from Jira STREAMPIPES-98. Original Jira may contain additional context.
Reported by: tex.

Error when installing W10

Dear team,

When I try to install ".bat" in W10 I receive the following error:

C:\Users\TA-INDUS-C41581\Desktop\streampipes-installer-master\streampipes\windows10>streampipes.bat start
& : No se puede cargar el archivo
C:\Users\TA-INDUS-C41581\Desktop\streampipes-installer-master\streampipes\windows10\streampipes.ps1. El archivo
C:\Users\TA-INDUS-C41581\Desktop\streampipes-installer-master\streampipes\windows10\streampipes.ps1 no está firmado
digitalmente. No se puede ejecutar este script en el sistema actual. Para obtener más información acerca de la
ejecución de scripts y la configuración de la directiva de ejecución, consulta about_Execution_Policies en
https:/go.microsoft.com/fwlink/?LinkID=135170.
En línea: 1 Carácter: 3

  • & 'C:\Users\TA-INDUS-C41581\Desktop\streampipes-installer-master\stre ...
  • + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess
    
    

I have tried to modify this policy but I receive this error:
Set-ExecutionPolicy : Windows PowerShell actualizó la directiva de ejecución correctamente, pero el valor queda
invalidado por una directiva definida en un ámbito más específico. Debido a esta invalidación, el shell mantendrá la
directiva de ejecución efectiva actual: RemoteSigned. Escriba "Get-ExecutionPolicy -List" para ver la configuración de
la directiva de ejecución. Para obtener más información, vea "Get-Help Set-ExecutionPolicy".
En línea: 1 Carácter: 1

  • Set-ExecutionPolicy "Unrestricted" -Scope CurrentUser -Confirm:$false
  •   + CategoryInfo          : PermissionDenied: (:) [Set-ExecutionPolicy], SecurityException
      + FullyQualifiedErrorId : ExecutionPolicyOverride,Microsoft.PowerShell.Commands.SetExecutionPolicyCommand
    
    

Could you help me?

REgards

Pipeline Log Button

The Pipeline Log button has no functionality

Imported from Jira STREAMPIPES-92. Original Jira may contain additional context.
Reported by: tex.

Dashboard without name

It should not be possible to create a dashboard without a name

Imported from Jira STREAMPIPES-94. Original Jira may contain additional context.
Reported by: tex.

Rename File-Download Files

Currently there is a static file name containing a space character is created. Change this to a more meaningful name (e.g. date_indexname) with no special characters

Imported from Jira STREAMPIPES-47. Original Jira may contain additional context.
Reported by: zehnder.

Notification Sink Placeholder

The notification sink placeholder are not working. #propertyName# are not replaced by the value of the property.

Imported from Jira STREAMPIPES-93. Original Jira may contain additional context.
Reported by: tex.

Data Sink for MQTT

Add a datasink for MQTT to module streampipes-sinks-broker-jvm.

 

Imported from Jira STREAMPIPES-71. Original Jira may contain additional context.
Reported by: zehnder.

User Feedback

Feedback button (top right) only opens a blank window (please see attachment)

Imported from Jira STREAMPIPES-83. Original Jira may contain additional context.
Reported by: tex.

Check licenses in UI module

Do a complete license check of all packages used in the StreamPipes UI module

Imported from Jira STREAMPIPES-5. Original Jira may contain additional context.
Reported by: riemer.

Deploy snapshots to Docker Hub

Snapshots of the backend and pipeline element containers should be published to Docker Hub to ease development of StreamPipes

Imported from Jira STREAMPIPES-10. Original Jira may contain additional context.
Reported by: riemer.

Dashboard widget Edit

When pressing the Edit button of a Dashboard Widget only a blank window opens

Imported from Jira STREAMPIPES-97. Original Jira may contain additional context.
Reported by: tex.

Processor: Merge two data streams by timestamp

A user can use this processor to merge two data streams by their timestamp.

The processor should be implemented in Java to also run on edge devices.

Imported from Jira STREAMPIPES-59. Original Jira may contain additional context.
Reported by: zehnder.

UI is not open source

I can not find UI code source? UI is not open source?Where is UI code source?thank you

Quick Edit Parameter Update

Element parameters are not saved when edit with quick edit, also __ with pressing the save button

Imported from Jira STREAMPIPES-91. Original Jira may contain additional context.
Reported by: tex.

Sink: HDFS

Description

Create a sink to store events into HDFS.

Input Requirements

  • None

Configurations

  • Host / Port
  • Folder Route
  • User / Password (optional)
  • Parameter for Buckets 

 

Imported from Jira STREAMPIPES-49. Original Jira may contain additional context.
Reported by: zehnder.

Frontend copy past mistake

Perhaps this copy past error can be corrected:
The first Reference Point Latitude should be Reference Point Longitude.
latitude-to-longitude

Pipeline cannot be saved

Reproduce:

  • Create Pileline
  • Add additional Element
  • Delete Element again

-> Pipeline cannot be saved

Imported from Jira STREAMPIPES-86. Original Jira may contain additional context.
Reported by: tex.

Fix Docker repo URLs in installer CLI

The env file of the CLI module of the installer links to some internal repos and need to be changed to a public Docker registry.

Imported from Jira STREAMPIPES-29. Original Jira may contain additional context.
Reported by: riemer.

Add processor to filter for numerical and text values

Currently, there are Numerical Filter and Text Filter. If a user wants to apply both, i.e. filter a field for a given string AND subsequently filter a numeric field based on a given threshold, these two PE's need to be concatenated to achieve this goal. In order to improve this common scenario there should be a new processor "Numerical Text Filter" to allow doing this in one step.

Imported from Jira STREAMPIPES-72. Original Jira may contain additional context.
Reported by: wiener.

Move CI to Apache infrastructure

Move the currently used Gitlab CI setup to Jenkins or Github Actions to make CI work in the Apache infrastructure

Imported from Jira STREAMPIPES-18. Original Jira may contain additional context.
Reported by: riemer.

Extract utils class for mqtt related configurations

We have multiple adapters that connect to a MQTT broker. Therefore, they need the same configuration parameters.

To avoid having to implement these for every adapter we need a Utils class which encapsulates the configuration. 

 

Imported from Jira STREAMPIPES-69. Original Jira may contain additional context.
Reported by: zehnder.

Remove dependency to kafka-rest

Kafka-rest is licensed under the Confluent Community license. Although not being used in the code directly, Kafka-rest ist currently required to get live data in the data stream preview. We need to find another way to show real-time previews.

Imported from Jira STREAMPIPES-9. Original Jira may contain additional context.
Reported by: riemer.

Data Sink for MQTT

Add a datasink for MQTT to module streampipes-sinks-broker-jvm.

 

Imported from Jira STREAMPIPES-71. Original Jira may contain additional context.
Reported by: zehnder.

Can't connect Pipeline elements

Reproduce:

  • Create Pileline
  • Leave Editor (e.g. go to Data explorer)
  • Go back to Editor 

-> sometimes elements cannot conntected anymore (see attachment)

Imported from Jira STREAMPIPES-85. Original Jira may contain additional context.
Reported by: tex.

Use internal file service in file adapters

As described in the thread: Re: STREAMPIPES-75: Extend data lake sink to store images on the mailing list. The uploaded files of the file stream and file set adapter should not be stored in the local container, instead they should be stored in the centralized file service..

 

 

 

Imported from Jira STREAMPIPES-80. Original Jira may contain additional context.
Reported by: zehnder.

Exception in adpaters when the date format is incorect

When a user marks a property as a timestamp, there is the possibility to provide a regex to convert the timestamp represented as a String into the internal representation of a UNIX timestamp in milliseconds. As long as the date in the events is formatted according to the regex this works properly. When there is an error in the data an exception is thrown and the adapter stops producing data. In this case the adapter should ignore the event instead and continue with the next event.

Imported from Jira STREAMPIPES-57. Original Jira may contain additional context.
Reported by: zehnder.

Dashboard without name

It should not be possible to create a dashboard without a name

Imported from Jira STREAMPIPES-94. Original Jira may contain additional context.
Reported by: tex.

Test Issue

This issue serves to identify the current issue ID ;-)

Considering integrating StreamPipes and DSS?

Apache StreamPipes is a very great project in the field of tream processing frameworks, and I think it is a good way to enhance the influence of our two projects is by integrating StreamPipes with DataSphere Studio.
DataSphere Studio is a one-stop data application development and management portal open source by WeBank, which meets the needs of the entire process of data application development from data exchange, desensitization and cleaning, analysis and mining, quality inspection, visual display, regular scheduling to data output.
Github address: https://github.com/WeBankFinTech/DataSphereStudio

Could not stop all pipeline elements

Hi, I have the problem that I can't stop a pipeline. Most likely I just used the Pipeline Editor wrong.
This is my pipeline:
pipeline
This is the error when I try to stop the pipeline:
error

Dashboard Widget without required Data

should not be possible to create a dashboard widget for a data stream that does not contain the required data.

For example, select a data stream without an image, create an image widget and save the dashboard -> the dashboard "crashes".

Imported from Jira STREAMPIPES-96. Original Jira may contain additional context.
Reported by: tex.

Pipeline Update blank info box

When you save an updated pipeline, an empty info box is displayed in the top right corner (see attachment).

Imported from Jira STREAMPIPES-89. Original Jira may contain additional context.
Reported by: tex.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.