Command line option to read query from file (feature request) about jello

[kellyjonbrazil]

I think I have this working pretty well now with -i:

% cat .jelloconf.py 
# from jello.lib import opts
# opts.mono = True

def darwin_compatible():
    result = []
    for entry in _.parsers:
      if "darwin" in entry.compatible:
        result.append(entry.name)
    return result
% jc -a | jello -ic 'darwin_compatible()' 
["airport","airport_s","arp","asciitable","asciitable_m","bluetoothctl","cbt","cef","cef_s","cksum","clf","clf_s","crontab","crontab_u","csv","csv_s","date","datetime_iso","df","dig","du","email_address","env","file","finger","git_log","git_log_s","git_ls_remote","group","hash","hashsum","history","hosts","id","ifconfig","ini","ini_dup","ip_address","iso_datetime","jar_manifest","jobs","jwt","kv","last","ls","ls_s","lsof","m3u","mount","netstat","openvpn","passwd","pci_ids","pgpass","ping","ping_s","pip_list","pip_show","plist","ps","rsync","rsync_s","semver","shadow","ssh_conf","sshd_conf","stat","stat_s","sysctl","syslog","syslog_s","syslog_bsd","syslog_bsd_s","time","timestamp","toml","traceroute","uname","uptime","url","ver","w","wc","who","x509_cert","xml","xrandr","yaml","zipinfo","zpool_iostat","zpool_status"]

I'll do more testing and add some unit tests before releasing.

[kellyjonbrazil]

I've added the following features to the dev branch which will be in the 1.6.0 release:

• Add the ability to directly use a JSON file or JSON Lines files as data input (-f)
• Add the ability to load a query from a file (-q)
• Add the empty data option (-e)
• Fix user-defined functions in ~/.jelloconf initialization file

[djczaski]

Looks good. I'll let you know if I find anything why using it.

[kellyjonbrazil]

Hi there - this is a good idea, but should already be supported with the -i option. This could be better documented, but can be found under the Advanced Usage section.

The problem is I seem to have broken this functionality while tightening up some things in the query environment. It should work like this (but does not right now):

% jc -a | jello -i 'darwin_compatible()'

If you have this in your ~/.jelloconf.py file:

def darwin_compatible(data=_):
    result = []
    for entry in data.parsers:
      if "darwin" in entry.compatible:
        result.append(entry.name)
    return result

I'll look into this and add some tests.

[djczaski]

Yes, I had tried the -i option but it didn't work. I was trying to debug a little before reporting it -- didn't know if it was broken or if it was me being a new user.

Personally, I think a -f program_file option would have value on it's own, much like sed and awk allow you to run programs from a file. It wasn't difficult to add in myself (except for the command line parsing). I hope you'll consider it at some point.

[kellyjonbrazil]

I have a little proof of concept working with the -i method. I have to change quite a bit of code since I locked down the scope of the exec call. Still thinking on how best to implement this and then the file method. It was easier before because I was not really limiting what could go into the exec call so the variables in the pyquery function were effectively shared.

[djczaski]

For the query, I did this:

diff --git a/jello/cli.py b/jello/cli.py
index 4b07197..da5b41d 100644
--- a/jello/cli.py
+++ b/jello/cli.py
@@ -122,7 +122,12 @@ def main(data=None, query='_'):
     long_options = {}

     for arg in sys.argv[1:]:
-        if arg.startswith('-') and not arg.startswith('--'):
+        if arg.startswith('-f'):
+            _, path = arg[2:].split('=')
+            with open(path, encoding="utf-8") as f:
+                query = f.read()
+
+        elif arg.startswith('-') and not arg.startswith('--'):
             options.extend(arg[1:])

[kellyjonbrazil]

I'm wondering if it makes sense to add file support as JSON input as well. That would make the -f option confusing, though.

$ jello -f file.json

Could maybe use something like -q for query file and -f for file input? Still considering options.

[djczaski]

-q query_file seems reasonable.

Maybe -d json_data or -j json_file.

[kellyjonbrazil]

released in v1.6.0