attempting to install first time about target-jsonl-blob

[EMCP]

im going to close this as I cannot dedicate time to sorting this out more... and focusing instead on using parquet

[EMCP]

I built the blob and now just.. taking a look how to run it properly.. wire in the config.. so thats cool.. one issue is at first run the help menu is expecting files which.. might be counter intuitive

 ./target-jsonl-blob --help
JSONLines Singer target for blob storages

Usage:
  target-jsonl-blob [flags]

Flags:
  -c, --config string   Config file
  -h, --help            help for target-jsonl-blob
  -i, --input string    Input file
2022/03/19 10:24:42 unable to decode into struct, Config File "config" Not Found in "[]"

[EMCP]

ah I see now in meltano I have the following in my config

loaders:
  - name: target-jsonl-blog
    namespace: target_jsonl_blog
    pip_url: target-jsonl-blog
    executable: target-jsonl-blog

i will move the executable somewhere and wire it into this loader .. and start debugging the docs.. for wiring this into S3.. but it would be good to maybe confirm what I should do installation wise (like are those errors expected and I should ignore them ? )

[EMCP]

okay nice,

I wired in the full target path.. and now seeing something that makes sense

$  meltano invoke  target-jsonl-blob
2022/03/19 11:02:49 Using config file /home/.meltano/run/target-jsonl-blob/target.60f9783b-b941-45ba-993a-c32a0d0eb7e4.config.json
2022/03/19 11:02:49 unable to decode into struct, bucket is required

EDIT: I now have a config that i think is valid .. but wondering where/how to wire in AWS credentials? or is this something I need to add in source?

  - name: target-jsonl-blob
    namespace: target_jsonl_blob
    executable: /home/target-jsonl-blob/target-jsonl-blob
    config:
      bucket: file:///home/meltano-project-ib/load

[edgarrmondragon]

Hi @EMCP sorry for the delay, I was out for a couple of weeks and this got buried below other notifications.

one issue is at first run the help menu is expecting files which.. might be counter intuitive

That looks like a 🐛: #4

I wired in the full target path.. and now seeing something that makes sense

Yup, the executable has to be an absolute path. I don't think you can plug the project path but that would be helpful, e.g. $MELTANO_PROJECT_ROOT/loaders/target-jsonl-blob.

wondering where/how to wire in AWS credentials? or is this something I need to add in source?

The options are

* Environment Variables
* Shared Credentials file
* Shared Configuration file (if SharedConfig is enabled)
* EC2 Instance Metadata (credentials only)

from https://docs.aws.amazon.com/sdk-for-go/api/aws/session/