toobaz / ipynb_output_filter Goto Github PK

View Code? Open in Web Editor NEW

105.0 105.0 29.0 15 KB

Script for stripping output from ipython/jupyter scripts

Python 100.00%

ipynb_output_filter's People

Contributors

Stargazers

Watchers

ipynb_output_filter's Issues

Committing changes triggers a bunch of NBFormatErrors.

When I try to commit changes, I get a bunch of NBFormatErrors.

Looks like it might be a difference between version 4 and version 3...?

I'm a long-time python/git user, but pretty new to notebooks, so help would be much appreciated.

Here's the traceback:

$ git commit -am "added some changes..."
Traceback (most recent call last):
  File "util/ipynb_output_filter.py", line 30, in <module>
    json_in = reads(to_parse, version)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 137, in reads
    return reads_json(s, **kwargs)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 90, in reads_json
    raise NBFormatError('Unsupported JSON nbformat version %s (supported version: %i)' % (nbf, 3))
IPython.nbformat.current.NBFormatError: Unsupported JSON nbformat version 4 (supported version: 3)
error: external filter util/ipynb_output_filter.py failed 1
error: external filter util/ipynb_output_filter.py failed
Traceback (most recent call last):
  File "util/ipynb_output_filter.py", line 30, in <module>
    json_in = reads(to_parse, version)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 137, in reads
    return reads_json(s, **kwargs)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 90, in reads_json
    raise NBFormatError('Unsupported JSON nbformat version %s (supported version: %i)' % (nbf, 3))
IPython.nbformat.current.NBFormatError: Unsupported JSON nbformat version 4 (supported version: 3)
error: external filter util/ipynb_output_filter.py failed 1
error: external filter util/ipynb_output_filter.py failed
[notebook-friendly Traceback (most recent call last):
  File "util/ipynb_output_filter.py", line 30, in <module>
    json_in = reads(to_parse, version)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 137, in reads
    return reads_json(s, **kwargs)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 90, in reads_json
    raise NBFormatError('Unsupported JSON nbformat version %s (supported version: %i)' % (nbf, 3))
IPython.nbformat.current.NBFormatError: Unsupported JSON nbformat version 4 (supported version: 3)
error: external filter util/ipynb_output_filter.py failed 1
error: external filter util/ipynb_output_filter.py failed
Traceback (most recent call last):
  File "util/ipynb_output_filter.py", line 30, in <module>
    json_in = reads(to_parse, version)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 137, in reads
    return reads_json(s, **kwargs)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 90, in reads_json
    raise NBFormatError('Unsupported JSON nbformat version %s (supported version: %i)' % (nbf, 3))
IPython.nbformat.current.NBFormatError: Unsupported JSON nbformat version 4 (supported version: 3)
error: external filter util/ipynb_output_filter.py failed 1
error: external filter util/ipynb_output_filter.py failed
30d6221] added changes...
Traceback (most recent call last):
  File "util/ipynb_output_filter.py", line 30, in <module>
    json_in = reads(to_parse, version)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 137, in reads
    return reads_json(s, **kwargs)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 90, in reads_json
    raise NBFormatError('Unsupported JSON nbformat version %s (supported version: %i)' % (nbf, 3))
IPython.nbformat.current.NBFormatError: Unsupported JSON nbformat version 4 (supported version: 3)
error: external filter util/ipynb_output_filter.py failed 1
error: external filter util/ipynb_output_filter.py failed
Traceback (most recent call last):
  File "util/ipynb_output_filter.py", line 30, in <module>
    json_in = reads(to_parse, version)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 137, in reads
    return reads_json(s, **kwargs)
  File "/Library/Python/2.7/site-packages/IPython/nbformat/current.py", line 90, in reads_json
    raise NBFormatError('Unsupported JSON nbformat version %s (supported version: %i)' % (nbf, 3))
IPython.nbformat.current.NBFormatError: Unsupported JSON nbformat version 4 (supported version: 3)
error: external filter util/ipynb_output_filter.py failed 1
error: external filter util/ipynb_output_filter.py failed
 2 files changed, 40 insertions(+), 6 deletions(-)

Empty output metadata fields "collapsed" and "scrolled"

Hi,

I have modified the script to reset the "collapsed" fields to false and delete the "scrolled" fields from cell metadata, to avoid detecting uncommited changes if you just collapsed some cells.

I think is not a crazy scenario, but maybe I'm missing how it breaks some other workflow...
I can upload and pull-request the changes if you think it's a good idea.

Thanks for maintaining the repo!

Configuration for specific .git repo

I work on several different machines where my repo is synced. I don't want to configure all machines separately.

I would like to have the option to clean the .ipynb from it's output in just this specific repo.

Could you please provide the settings how I could do this just for one specific repository?

Also, does this work for v4 ipynb?

I suppose its just:

git config core.attributesfile ~/myrepo/.gitattributes
git config filter.dropoutput_ipynb.clean ~/myrepo/script/ipynb_output_filter.py
git config filter.dropoutput_ipynb.smudge cat

cleanup widget states from ipynb

I need to update this script to cleanup the state left by widgets (from the ipywidget project) as well..

I believe it would make sens to integrate this feature to you repo, are you interested in a pull request ?

trim `language_info.version`

Sorry for multi-posting, but I thought it was better to keep things in different issues.

The problem I'm trying to solve appears if working with computers that have different (but compatible) versions of the programming language (say python 2.7.11 in computer A and 2.7.9 in computer B). Then, just by running a notebook with a computer different from the one to run and commit the file, git/mercurial will detect uncommitted changes.

I'm thinking on processing the metadata.language_info.version to keep only major and minor version number, and discard the patch version number (http://semver.org/), so that 2.7.11 would become 2.7. It sounds weird, but in principle patch version should not affect backwards compatibility, so it should be safe. Still, I'm not sure it is a good thing for the general use case. What do you think?

Shall I pull-request or keep it as a personal hack?

No module named IPython.nbformat.current

I thought I would completely start a new repo where I don't save outputs.

So I did git init and applied the filters, but now I get the following error:

  File "/Users/basnijholt/orbitalfield/scripts/ipynb_output_filter.py", line 16, in <module>
    from IPython.nbformat.current import reads, write
ImportError: No module named IPython.nbformat.current
error: external filter ~/orbitalfield/scripts/ipynb_output_filter.py failed 1

Any idea?

Reconsider whether to strip out "id" field

The commit db50349 strips "id" field out.

But GitHub fails to recognize the notebook if the "id" field is null.

Without "id": https://github.com/AllanChain/fastbook-but-jax/blob/157567d/mnist.ipynb
With "id": https://github.com/AllanChain/fastbook-but-jax/blob/0e1d9b8/mnist.ipynb

Also, if I try to delete the "id" field instead of setting it to None, nbformat will automatically re-add the field. So I don't think stripping out the "id" field is a good idea.

Furthermore, the "id" field is required to be a string in the schema:

Shebang still uses Python 2

The shebang (#!/usr/bin/env python) still refers to python, which at least on Linux generally refers to Python 2 as far as I know. I don’t know whether using python3 would work in all contexts here, but I had to change this to get things working. At the very least there could be a warning in the instructions.

(Also, the Python 2 hooks can arguably be retired now, as it is officially dead.)

Specify Python 2 and 3 support ?

Don't know if this would be a good idea - but might be helpful to some to state it supports both Python 2 and 3. Feel free to close if not, thanks! 👍

script not working

Hi,

Thanks a lot for the very clear description of the steps and the script. However, it doesn't seem to be working for me. I have enabled it globally, and yet, it detects changes when just the "execution_count" in my ipynb in my repo changes. I have followed your steps exactly as stated. Please provide some inputs to fix it.

toobaz / ipynb_output_filter Goto Github PK

ipynb_output_filter's People

Contributors

Stargazers

Watchers

Forkers

ipynb_output_filter's Issues

Committing changes triggers a bunch of NBFormatErrors.

Empty output metadata fields "collapsed" and "scrolled"

Configuration for specific .git repo

cleanup widget states from ipynb

trim `language_info.version`

No module named IPython.nbformat.current

Reconsider whether to strip out "id" field

Shebang still uses Python 2

Specify Python 2 and 3 support ?

script not working

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Jobs