Comments (24)
@sharkdp What do you think if we can add du or something like this?
In which sense? Like a filesize :: Path -> Int
function?
By the way, I just realized that the coreutils du
commands works very nicely with functools:
> du | filter -c1 greater_than 8000 | map -c2 abspath
8340 /tmp/demo/folder/subdirectory
8408 /tmp/demo/folder
8480 /tmp/demo
from shell-functools.
I think array-reverse can be useful. Not sure if we need a string-reverse function.
from shell-functools.
Do you know jq
? I think this could probably play well in combination with shell-functools.
from shell-functools.
String
- search and match based on regex
- Add regex support for replace input
- Add function endwith a pattern
Arithmetic
- Add mod, pow, div and log etc. functions
Filesystem
- Add function mv to move and rename files
- More functions to filter files based on file mode bits and ownship
from shell-functools.
@LeeLeeee Sounds great. Especially the mv
function, which will be really useful (we should add cp
as well)
from shell-functools.
@sharkdp What do you think if we can add du or something like this?
from shell-functools.
Yeah, more precisely it is filesize :: Path -> String
.
$ find . | map basename | filter startswith test_ | map filesize mb
test_file1 1MB
test_file2 5.2MB
It is fantastic that linux commands are work very well with these functions. 👍
from shell-functools.
filesize
is now supported.
from shell-functools.
-
fst
andsnd
for array-index access (alias forindex 1
andindex 2
) -
replicate
-
reverse
from shell-functools.
Does split currently work with variable size of whitespace between words? if not, does it make sense to add it? Something like below by default if no separator is provided, then it would be a nice replacement for awk
➜ ~ echo "a b ccc " | ~/repos/shell-functools/ft/map split | map at 2
ccc
from shell-functools.
Does split currently work with variable size of whitespace between words?
It does not (yet). We could either provide a split_whitespace
function which would simply call .split()
without argument and/or provide a split_regex
function which could be called with map split_regex '\s+'
.
from shell-functools.
The reverse
function, do you mean on array level, or on string level?
from shell-functools.
Hey @sharkdp, what about creating a max
as well as min
function in shell-functools? I can work on something if there is interest.
max :: Array -> Int
find . -name cnt.txt | xargs cat | max
from shell-functools.
@guilhermeleobas Sounds great!
from shell-functools.
These days, new unix/linux CLI tools are coming out supporting JSON outputs. Would it make sense to create a json function that can parse JSON outputs?
from shell-functools.
These days, new unix/linux CLI tools are coming out supporting JSON outputs. Would it make sense to create a json function that can parse JSON outputs?
I've been thinking about JSON support in the past, so this sounds interesting. What would this function do, exactly?
from shell-functools.
I can imagine there would a command named json
which could be used like this:
$ curl http://echo.jsontest.com/key1/value1 | json key1
The curl
command outputs {"key1": "value1"}
. The whole command would output value1
. The argument to json
command would specify the path to the value you want. It could support the full json path spec or just a small part of it.
And then there would also a function called json
which could be used like this:
$ curl <url that returns json in each line> | map json <json path>
In this case, the assumption is that the curl
outputs a json-encoded string in each line, which is also a proposed standard format called jsonlines
. And the map will transform each line with the json function.
$ <command that outputs json> | json <path to an array of objects> | map json <path in each object>
In the above command, the json
command narrows data down to an array of objects, and then map json
transforms each json into something. I think the path you specify could be some kind of template so that you can transform json into anything, which sounds a bit too ambitious.
from shell-functools.
@sharkdp jq
seems to cover my use cases. Cool, thanks.
from shell-functools.
More functions that should be implemented:
-
div
/divide
-
sum
/product
(working on arrays) -
matches
=> similar tostarts_width
/ends_with
from shell-functools.
Offering
For README.md:
### Usage of `lambda`
The `lambda` *function* allows you to create custom functions using a
definition string to Python's *lambda* function.
``` bash
> seq 3 | map lambda 'x : x+x'
2
4
6
> echo -e 'a:b\n2:4' | map split ':' | map lambda 'x, y : x+y'
ab
6
> seq -3 3 | map lambda 'x : abs(x)'
3
2
1
0
1
2
3
> echo -e "uppercase\nLOWERCASE\ncAMELcASE" | map lambda "x : x.swapcase()"
UPPERCASE
lowercase
CamelCase
For ft/ft/functions.py:
@register("lambda")
@typed(None, None)
def ft_lambda(definition, inp):
_lambda = 'lambda ' + definition.value
if type(inp) == list:
args = list(map(lambda v: v.value, inp))
return [TypedValue(str((eval(_lambda))(*args)), T_STRING)]
else:
return (eval(_lambda))(inp)
from shell-functools.
Offering
For README.md:
### Usage of `shell`
The `shell` *function* allows you to create custom functions using arbitrary
*shell* commands. If the command string contains pairs curly braces `{}`, the
arguments to the function are formatted into it, otherwise they are appended
to it.
``` bash
> /usr/bin/ls -1 ft/ft/*.py | map shell 'grep -H -c def' | map split ':' | map format '{} has {} functions'
ft/ft/command.py has 14 functions
ft/ft/error.py has 1 functions
ft/ft/functions.py has 60 functions
ft/ft/__init__.py has 0 functions
ft/ft/internal.py has 4 functions
ft/ft/termcolor.py has 2 functions
ft/ft/test_command.py has 5 functions
ft/ft/types.py has 21 functions
ft/ft/version.py has 0 functions
> echo -e 'a\tb\n2\t4' | map shell 'echo "{} and {}"'
a and b
2 and 4
Total lines of text
find | filter is_file | map shell 'cat {} | wc -l' | foldl1 add
5040
For ft/ft/functions.py:
@register("shell")
@typed(None, None)
def shell(command, inp):
if type(inp) == list:
args = list(map(lambda v: str(v.value), inp))
else:
args = [str(inp)]
command = command.value
if '{}' in command:
command = command.format(*args)
else:
command += ' '+' '.join(args)
result = os.popen(command).read().rstrip('\r\n')
if type(inp) == list:
return [TypedValue(result, T_STRING)]
else:
return result
from shell-functools.
Offering
For README.md:
### Usage of `regex`
The `regex` *function* permits filtering based on Python supported regular
expressions. For case-insensitive filtering, use the `(?i:...)` syntax:
``` bash
> /usr/bin/ls | filter regex '(?i:[a-c]+)'
LICENSE
make
path
README.md
For ft/ft/functions.py:
import re
@register("regex")
@typed(T_STRING, T_BOOL)
def contains(pattern, inp):
pattern = dynamic_cast(T_STRING, pattern).value
return re.search(pattern, inp)
from shell-functools.
Offering of new command array
to create or flatten arrays
For README.md:
### Usage of `array`
The `array` command is used to take *lines of input*, and convert them into an
`array`.
``` bash
> seq 3 | array
1 2 3
Given an numerical argument, array
will make arrays with that number of
columns
:
> seq 12 | array 5
1 2 3 4 5
6 7 8 9 10
11 12
Given an argument of '0' (or a non-numerical argument), array
will convert
arrays back into lines of input:
> echo -e 'aa\tbb\tcc\tdd' | array 0
aa
bb
cc
dd
For ft/array (new):
#!/usr/bin/bash
if [[ -z $1 || $1 -gt 0 ]]; then
if [[ -n $1 ]]; then
max=$1
else
max=1000000000000000
fi
cnt=0
while read; do
if [[ $cnt -gt 0 ]]; then
result+=$'\t'
fi
result+=$REPLY
(( cnt++ ))
if [[ $cnt -eq $max ]]; then
echo "$result"
result=
cnt=0
fi
done
# Remaining partial line
if [[ -n $result ]]; then
echo "$result"
fi
else
# array 0
sed 's@\t@\n@g'
fi
# EOF
from shell-functools.
hi! can u expand on that? like what would these do or what would be their input be like.
div
/divide
sum
/product
(working on arrays)
- @ sharkdp at #7 (comment)
from shell-functools.
Related Issues (20)
- Handling of type-inference errors (file named `3`)? HOT 1
- Add take_while command
- Add shell-completion for available function names HOT 3
- Support for 'flip' HOT 2
- Commands not installed HOT 3
- Problem installing in Python3.6 env HOT 3
- Support for rename operations HOT 1
- Numeric file name HOT 1
- Custom functions HOT 3
- Installation error with pip3 HOT 6
- Predicates negation HOT 2
- filtering in a stream doesn't work as expected HOT 7
- Issues with filename consist of only numbers like '123' HOT 1
- is_link does not detect links HOT 4
- Arrays with ints is broken HOT 1
- Offering: Desending sort_by HOT 1
- Offering: Update the `run` function to allow for command line arguments HOT 1
- Offering: Add array support to `format` function HOT 1
- Offering: Allow `join` function to work on a single column HOT 1
- Maintenance status of this project HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from shell-functools.