alecthomas / pawk Goto Github PK

E.g.

$ printf 'foo\nbar' | pawk -Bt

foo
bar

I was just trying out pawk when I noticed this.

When I try to use a statement in the line action of pawk I get a syntax error. To reproduce just run a line taken direct from the README of pawk

pawk -B c=0 -E c 'c += os.stat(f[0]).st_size'

This results in this error:

File "/usr/local/lib/python2.7/site-packages/pawk.py", line 41, in _compile
self._codeobj = compile(self.cmd, 'EXPR', 'exec' if statement else 'eval')
File "EXPR", line 1
c += os.stat(f[0]).st_size
^
SyntaxError: invalid syntax

Any statement using "=" in the line action fails similarly. Statements seem to work in -B and -E actions, but only expressions work for line actions.

This is under python 2.7.5. Pawk does not report a version.

The newly added feature -H is great! However, it is even better if a user is allowed to process the header row as well. For example, in your example of

count name
12 bob
34 fred

pawk -H '"%s is %s" % (name, count)' < input.txt produces the following results.

name is count
bob is 12
fred is 34

Or at least, there should be an option allowing users to do so. One simple use case is to use pawk to extract columns from a text file. Users often want to keep the header row in this situation.

The PyPI release is at v0.8.1. Looking at issue #22 and the head commit, I don't think there were any changes to the actual pawk.py script. However, I am trying to add pawk to Homebrew PR, and it would be helpful if the two sources were in sync.

i.e., column names? I want to select columns by name.

Is it possible to invoke pawk from a python script, without using subprocess.run?

I am imagining something like:

import pawk

my_pawk_program = ' .... '

results = pawk.run(my_pawk_program)

I only tested Poetry build with my patch to pyproject.toml but made no further tests.

I added no scripts entries, thus no pawk script is installed 😕. Moreover, the bin directory containing python and hermit binaries is unnecessary.

I'll have a look and submit another PR...

This seems really cool...any chance for supporting in-place editing of files? I understand that awk doesn't support in-place editing but am used to the perl -i option and find it super-handy.

The pawk version on PyPI doesn't include #16 and, unfortunately, doesn't work out of the box with Python 3.8-3.10.

change:
echo '1 2' | pawk 'int(f[0])<=1'

to:
echo '1 2' | pawk 'f[0]<=1'

We do not need to make the conversion every time, especially when we have many operations.

Thanks much!

Hello, I'm building pawk on an Arch Linux machine with packages:
pawk v.0.7.0
pypandoc 1.8.1
The build process halts on the following error:

Traceback (most recent call last):                                                                  
  File "/home/gaetan/.cache/pikaur/build/python-pawk/src/pawk-0.7.0/setup.py", line 16, in <module> 
    long_description = pypandoc.convert('README.md', 'rst')                                         
AttributeError: module 'pypandoc' has no attribute 'convert'                                        

I have also tried pip install pawk (pawk version is 0.6.5) and building git zip file with the same result.

Verification: the pypandoc I use has convert_file and convert_text methods, but no convert

>>> dir(pypandoc)
['DEFAULT_TARGET_FOLDER', 'Iterable', 'Union', '__all__', '__author__', '__author_email__', '__builtins__', '__cached__', '__classifiers__', '__description__', '__doc__', '__file__', '__license__', '__loader__', '__name__', '__package__', '__pandoc_path', '__path__', '__python_requires__', '__setup_requires__', '__spec__', '__url__', '__version', '__version__', '_as_unicode', '_check_log_handler', '_classify_pandoc_logging', '_convert_input', '_ensure_pandoc_path', '_get_base_format', '_get_pandoc_version', '_identify_format_from_path', '_identify_input_type', '_identify_path', '_is_network_path', '_validate_formats', 'absolute_import', 'cast_bytes', 'cast_unicode', 'clean_pandocpath_cache', 'clean_version_cache', 'convert_file', 'convert_text', 'download_pandoc', 'ensure_pandoc_installed', 'ensure_pandoc_maximal_version', 'ensure_pandoc_minimal_version', 'get_pandoc_formats', 'get_pandoc_formats_pre_1_18', 'get_pandoc_path', 'get_pandoc_version', 'glob', 'handler', 'logger', 'logging', 'normalize_format', 'os', 'pandoc_download', 'print_function', 'py3compat', 're', 'string_types', 'subprocess', 'sys', 'tempfile', 'textwrap', 'url2path', 'urlparse', 'with_statement']
>>> 

I tried to use convert_file in place of convert but could not make setup.py run without error.

n - The current 1-based line number.

This seems a little bit confusing as the field index is 0-based. I suggest make them consistent. It is probably better to use 0-base index as that's what Python does.

Feature request to enable things like:

cat /etc/hosts | pawk '!/^#/ and !/^127'

This would require a whole new way of parsing. It should parse it as a python expression with some syntactic sugar for regex matches.

For that easier identification of version installed.

Hi,

Can you please add a feature to evaluate action(s) on each line before the output evaluation. For example, if I want to exchange two columns, I have to list the order in full, in the current implementation.

printf '%s ' {1..5} | pawk 'f[0],f[2],f[1],f[3],f[4]'
1 3 2 4 5

something like this:

printf '%s ' {1..5} | pawk -a 'f[2],f[1] = f[1],f[2]' 'f'

This is very convenient if we have too many columns. Or we just want to have a minor modification on each line and output results.

Thanks much!

Pawk crashes with, as far as I can tell, any pattern and expression if running on python 3.8.

~/pawk >>> echo '1 1' | python3.7 pawk 'f[0]'                                                ±[●][master]
1
~/pawk >>> echo '1 1' | python3.8 pawk 'f[0]'                                                ±[●][master]
Traceback (most recent call last):
  File "pawk", line 5, in <module>
    main()
  File "/home/archer/pawk/pawk.py", line 246, in main
    run(sys.argv, sys.stdin, sys.stdout)
  File "/home/archer/pawk/pawk.py", line 233, in run
    actions = [Action.from_options(options, arg) for arg in args]
  File "/home/archer/pawk/pawk.py", line 233, in <listcomp>
    actions = [Action.from_options(options, arg) for arg in args]
  File "/home/archer/pawk/pawk.py", line 88, in from_options
    return cls(pattern=pattern, cmd=cmd, have_end_statement=(options.end is not None), negate=negate, strict=options.strict)
  File "/home/archer/pawk/pawk.py", line 83, in __init__
    self._compile(have_end_statement)
  File "/home/archer/pawk/pawk.py", line 96, in _compile
    self._codeobj = compile_command(self.cmd)
  File "/home/archer/pawk/pawk.py", line 65, in compile_command
    return compile(tree, 'EXPR', 'exec')
ValueError: Name node can't be used with 'None' constant