cloudant-labs / data-flow-examples

Examples extracting data from Cloudant and writing it elsewhere

data-flow-examples

A series of data flow example tutorials using various technologies to extract data from IBM Cloudant and write the data elsewhere.

Examples to IBM Db2 Warehouse on Cloud

These examples use Db2 Warehouse on Cloud as a destination.

N.B. None of these examples are a direct replacement for the deprecated Cloudant warehouse integration.

The examples can be used as a starting point to develop application specific data flows, but will need to be modified to meet your application requirements. For example care should be taken around handling:

• document updates
• document deletions
• schema modifications/table alteration
• nested JSON structures and other data type limitations

1. IBM Cloud Functions
2. Apache Spark on Watson Studio
3. Node-RED on IBM Cloud
4. IBM Message Hub
5. IBM Streams Designer (Beta) on Watson Studio

Learn more about the differences between these technologies for extracting Cloudant documents and writing the data to a Db2 Warehouse on Cloud table.

data-flow-examples's Issues

How to automatically trigger python script after changes made in cloudant ?

Cloudant Warehouse Create warehouse option not available ?

" Create warehouse option is not available," I think after depreciation this option has been removed.
what is an alternative option available to create a warehouse with Cloudant?

Cloudant_to_Db2_streams_flow.stp

Could you please guide the purpose of .stp file format and from where it is generated.

How to get all database from cloudant ?

from given code below rather than getting only animaldb i want to access all available databases through the loop, how can I do that?

Rename IBM Message Hub -> IBM Event Streams

The Message Hub service is changing name the tutorials should be updated to reflect this change.

• The tutorial where you found the bug or issue.

  https://github.com/cloudant-labs/data-flow-examples/blob/master/message-hub/README.md

• The step you were performing when you found the issue.

  All steps that refer to Message Hub

How can I recover deleted document from cloudant database ?

How to make custom table in Cloudant to Db2 in Spark using Watson Studio ?

I want to make a table in Db2 warehouse for the give (document screenShot) in Spark using Watson studio.

The table cloumn name should be the content(invNumber,salePrice..) json object in 'totalInvoice' JsonArray.

Secondly:
If I want to do the same job with node-red how can I do that?