elasticio / mssql-component

Microsoft SQL Server integration component for elastic.io platform

mssql-component

elastic.io integration component for Microsoft SQL Server

mssql-component

MSSQL Component component for the elastic.io platform

Environment Variables

No required Environment Variables.

Before you Begin

Before you can deploy any code into elastic.io you must be a registered elastic.io platform user. Please see our home page at http://www.elastic.io to learn how.

Getting Started

Authentication

You may use following properties to configure a connection:

other types of configuration parameters are also supported, more infromation and samples you can find here

SELECT Trigger and Action

With this action you may fetch data out of the database, e.g. using SELECT statement.

This trigger & action has no limitations on the number of rows so you may expect to get all of these via sequential fetching that is implemented within the node.js mssql driver.

Polling

Component will remember last execution timestamp and let you build queries on it:

select * from Leads where Created >= '%%EIO_LAST_POLL%%'

where just before executing the statement the %%EIO_LAST_POLL%% will be replaced with ISO Date of the last execution, for example 2017-08-25T07:43:48.127Z. During the first execution, date will be equal to the bigging of time - 1970-01-01T00:00:00.000Z.

INSERT/DELETE/UPDATE Action

You may use this action to do the operations that are not producing output rows but do the database manipulations, e.g. INSERT, UPDATE or DELETE statements. Internally we use prepared statements, so all incoming data is validated against SQL injetion, however we had to build a connection from JavaScript types to the MSSQL data types therefore when doing a prepared statements you would need to add :type to each prepared statement variable.

For example if you have a following SQL statement:

INSERT INTO 
  Test2.dbo.Tweets 
(Lang, "Text", id, CreatedAt, Username, ScreenName) 
VALUES 
(@lang, @text, @id, @created_at, @username, @screenname)

you should add :type to each @parameter so your SQL query will looks like this:

INSERT INTO 
  Test2.dbo.Tweets 
(Lang, "Text", id, CreatedAt, Username, ScreenName) 
VALUES 
(@lang, @text, @id:bigint, @created_at:date, @username, @screenname)

Following types are supported:

• string (also default type if type is omitted)
• number (will be converted to MSSQL int)
• bigint
• boolean (will be converted to MSSQL bit)
• float
• date (will be converted to DateTime2)
• money

more details can be found here

Component supports dynamic incomig metadata - as soon as your query is in place it will be parsed and incoming metadata will be generated accordingly.

License

Apache-2.0 © elastic.io GmbH

mssql-component's Issues

MSSQL. Update component to new sailor version / use new logger functionality / dockerize

DoD
Development:

• use new logger
• component should be dockerised, add: "buildType":"docker" line into component.json
• sailor version is changed from 2.2.1 to 2.6.1
• CHANGELOG.md is updated

QA:

• Smoke test is finished
• Regression testing is finished (optional)
• Support is asked to deploy to the production stage (app.elastic.io) and/or another required stage
• Component is actually deployed to the production stage (app.elastic.io) and/or another required stage
• Smoke after deploying to the production stage

Smoke test checklist

• Push to platform
• Retrieve sample
• Retrieve sample on LA (if possible)
• Run keen flow
• Run realtime flow

Update Sailor to 2.6.7

PR #17

Investigate if it's possible to deprecate this component. If yes - depricate

It's quite old component which normally can be replaced with JDBC

images

For images

Revitalize component

Description:

• 1.
  Trigger is missing input fields, so on retrieve sample it throws an error:

Error! Cannot read property 'query' of undefined

• 2.
  Seems that INSERT/UPDATE/DELETE action is unable to update and delete as it throws errors on valid update and delete requests:

Requests:

• DELETE FROM Employee WHERE id=1234
• UPDATE Employee SET Designation = "UPDATE-test" WHERE EmpID = 123;

Error:
TypeError: vars is not iterable
at EventEmitter.getMetaModel (/home/apprunner/lib/actions/insert.js:101:29)
at /home/apprunner/node_modules/elasticio-sailor-nodejs/lib/service.js:225:62
at new Promise ()
at executeMethod (/home/apprunner/node_modules/elasticio-sailor-nodejs/lib/service.js:218:20)
at _fulfilled (/home/apprunner/node_modules/q/q.js:834:54)
at /home/apprunner/node_modules/q/q.js:863:30
at Promise.promise.promiseDispatch (/home/apprunner/node_modules/q/q.js:796:13)
at /home/apprunner/node_modules/q/q.js:556:49
at runSingle (/home/apprunner/node_modules/q/q.js:137:13)
at flush (/home/apprunner/node_modules/q/q.js:125:13)

• 3
  No Error thrown in INSERT/UPDATE/DELETE and SELECT actions on invalid query
  STR:

1. query: INSERT INTO Employee123 (EmpID,EmpName, JoingTimeStamp) VALUES (@EmpId:number,@EmpName:string, @JoingTimeStamp:date)
2. Retrieve sample

ER: error thrown
AR: sample retrieving is stuck

Meta Data Refresh

On the Insert/Update Action whenever you type in the field there the platform is always trying to refresh/validate some meta data or something.

It locks the UI and appears to perform no useful function.

Suggestion: remove the OnKeyUp/Down refresh and add a "refresh" button... or just completely remove.

SQL SELECT Statement trugger should throw an error when Statement is not `Select` and must not insert/modify data

Description Trigger throws an error Component did not send any data during timeout. when INSERT Statement is in input but actually data is added into DB

STR:

1. SQL SELECT Statement input:

INSERT INTO Employee (EmpID, EmpName, JoingTimeStamp)
VALUES (13251, 'try to insert', '2019-10-22T12:24:18.920Z');

2. Retrieve sample

AR:
Error thrown: Error! Component did not send any data during timeout. but row is actually inserted
Logs:

Starting your Docker container ...
[INFO  tini (1)] Spawned child process '/run.sh' with pid '6'
Starting application in apprunner...
Will try to start app at /home/apprunner
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
node: /usr/lib/libstdc++.so.6: no version information available (required by node)
Connecting to the database
Connection established
Checking if incoming messages is lightweight...
Last polling timestamp=1970-01-01T00:00:00.000Z
Query execution completed, affected=1

ER:
Error that only SELECT Statement is supported thrown
No data affected (nothing created/updated)

BULK Upload support

Sometimes it's important to load data not sequentially but as a BULK. Currently INSERT action inserts rows one-by-one therefore not efficient with large amounts of data.

Outdated version of the component is on the platform

There was implemented user friendly approach for configuration of a connection in PR #7.

Users don't see changes, because they hadn't been deployed. These changes should be pushed to elastic.io platform.

Timeout when entry is not found in the DB

Steps to reproduce:

1. Enter valid credentials
2. Select 'SELECT' action
3. Input query like 'SELECT * FROM Employee WHERE id=444', where Employee - existing table, id value should not exist in the table.

Expected result:
Message like 'nothing found' is shown or error is thrown

Actual result:
Error!
Component did not send any data during timeout.

Improved account configuration UI

Currently one have to enter an MSSQL URI as a string like:

mssql://username:password@localhost/database

Issues with it:

• Not very user friendly
• Password is visible in UI and not protected

However, advantages are that many possible settings could be build in as encoded in URI. We need t good solution to combine streights but not weaknesses.

Usage of mssql component

Is this component usable with a mysql database component? @