Comments (16)
This should also include filesize for modified files at the least.
from deltacode.
@majurg 2 questions:
-
You're referring to the
Delta.to_dict()
method, right? -
What info do you want to display (a) in the CSV output and (b) in the JSON output?
from deltacode.
- Yes
- We can start with Filename, filetype, and Size for now in both.
from deltacode.
Thanks @majurg . 👍
from deltacode.
Just saw another use case for this. OLD codebase was a Development Codebase with no build artifacts. NEW codebase included build artifacts (.o object files and similar) which show up as Added. There were many hundreds of these so it would have useful to have filetype data that would allow you to quickly confirm that. In this case the Added files should all be some binary format.
from deltacode.
@majurg Good use case -- thanks, Steven.
from deltacode.
@mjherzog -- My mistake, noticed just now that the author was mjherzog not majurg. Thank you, Michael!
from deltacode.
@majurg Working in VSCode this morning, I've begun encountering a series of internal errors when running py.test
, e.g., INTERNALERROR> IOError: [Errno 0] Error
. (That's just the last line of the error output.)
I began with the develop
branch, did a git pull
, cleaned and configured, then checked out a new branch. First, ran py.test
-- all 103 passed. Then made a few changes to Delta.to_dict()
, liked the output, ran py.test
, and started encountering these errors. Even after discarding all changes and running py.test
, the internal errors persist. Same result after rebooting.
Do you have any familiarity with this sort of problem? Would it be helpful if I paste the entire error output? Might an uberconf help track down what's going on?
from deltacode.
Not sure, maybe see if that happens on a freshly cloned detlacode repo?
from deltacode.
@majurg I deleted my local DeltaCode repo, cloned the repo, cleaned and configured, opened the develop
branch in VSCode. Ran source bin/activate
. First time I ran py.test
, all 103 tests passed. Ran it again, got the now-familiar error message (it occurs at different points in the test suite):
(deltacode)
JMH@JMH-T510 MINGW64 C:/code/nexb/dev/deltacode (develop)
$ py.test
============================= test session starts =============================
platform win32 -- Python 2.7.12, pytest-3.1.0, py-1.4.33, pluggy-0.4.0 -- c:\code\nexb\dev\deltacode\scripts\python.exe
cachedir: .cache
rootdir: C:\code\nexb\dev\deltacode, inifile: setup.cfg
plugins: xdist-1.16.0
collected 103 items
tests/test_cli.py::TestCLI::test_generate_csv_added PASSED
tests/test_cli.py::TestCLI::test_generate_csv_modified PASSED
tests/test_cli.py::TestCLI::test_generate_csv_modified_new_license_added PASSED
tests/test_cli.py::TestCLI::test_generate_csv_modified_new_license_added_low_score PASSED
tests/test_cli.py::TestCLI::test_generate_csv_removed PASSED
tests/test_cli.py::TestCLI::test_generate_csv_renamed PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_None_paths PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_abcm_aligned PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_align_scan_zlib_alignment_exception PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_delta_len_error PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_ecos_failed_counts_assertion PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_empty_paths PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_file_added PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_file_modified PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_file_removed PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_file_renamed PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_identical PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_invalid_paths PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_license_modified PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_license_modified_low_score PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_no_license_changes PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_no_license_key_value PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_empty PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_original_path_added1 PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_original_path_dropbear PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_original_path_full_root PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_original_path_openssl PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_original_path_zlib PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_simple_file_added PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_simple_file_modified PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_to_dict_simple_unmodified PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_zlib_unaligned PASSED
tests/test_deltacode.py::TestDeltacode::test_DeltaCode_zlib_unaligned_same_base_path PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_create_object_added PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_create_object_empty PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_create_object_modified PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_create_object_removed PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_create_object_unmodified PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_None_files PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_missing_diff_low_score_new PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_missing_diff_low_score_old PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_new_no_license_info PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_no_diff PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_no_diff_multiple_keys PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_no_diff_multiple_keys_low_score_new PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_no_diff_multiple_keys_low_score_old PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_old_no_license_info PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_one_None PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_single_diff PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_license_diff_single_diff_multiple_keys PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_added PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_empty PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_modified PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_modified_license_added PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_modified_license_added_low_score PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_modified_no_license_changes PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_modified_no_license_key PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_removed PASSED
tests/test_deltacode.py::TestDeltacode::test_Delta_to_dict_unmodified PASSED
tests/test_deltacode.py::TestDeltacode::test_align_and_index_scans PASSED
tests/test_models.py::TestModels::test_File_create_object PASSED
tests/test_models.py::TestModels::test_File_create_object_license_missing PASSED
tests/test_models.py::TestModels::test_File_create_object_license_none PASSED
tests/test_models.py::TestModels::test_File_create_object_license_one PASSED
tests/test_models.py::TestModels::test_File_empty PASSED
tests/test_models.py::TestModels::test_File_size_difference PASSED
tests/test_models.py::TestModels::test_File_to_dict_empty PASSED
tests/test_models.py::TestModels::test_File_to_dict_simple PASSED
tests/test_models.py::TestModels::test_File_to_dict_simple_w_license PASSED
tests/test_models.py::TestModels::test_License_object_empty PASSED
tests/test_models.py::TestModels::test_License_object_simple PASSED
tests/test_models.py::TestModels::test_License_to_dict_empty PASSED
tests/test_models.py::TestModels::test_License_to_dict_simple PASSED
tests/test_models.py::TestModels::test_Scan_None_path PASSED
tests/test_models.py::TestModels::test_Scan_empty_json PASSED
tests/test_models.py::TestModels::test_Scan_empty_path PASSED
tests/test_models.py::TestModels::test_Scan_empty_text PASSED
tests/test_models.py::TestModels::test_Scan_get_options_license_no PASSED
tests/test_models.py::TestModels::test_Scan_get_options_license_yes PASSED
tests/test_models.py::TestModels::test_Scan_index_files_key_missing PASSED
tests/test_models.py::TestModels::test_Scan_index_files_key_name_multiple_copies PASSED
tests/test_models.py::TestModels::test_Scan_index_files_key_path PASSED
tests/test_models.py::TestModels::test_Scan_index_files_key_sha1_foo PASSED
tests/test_models.py::TestModels::test_Scan_index_files_key_sha1_multiple_copies PASSED
tests/test_models.py::TestModels::test_Scan_index_files_large_dropbear_key_name PASSED
tests/test_models.py::TestModels::test_Scan_index_files_large_dropbear_key_path PASSED
t
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\main.py", line 105, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\main.py", line 141, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\main.py", line 164, in pytest_runtestloop
INTERNALERROR> item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
INTERNALERROR> return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 254, in _wrapped_call
INTERNALERROR> return call_outcome.get_result()
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 280, in get_result
INTERNALERROR> _reraise(*ex) # noqa
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 265, in __init__
INTERNALERROR> self.result = func()
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
INTERNALERROR> return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 254, in _wrapped_call
INTERNALERROR> return call_outcome.get_result()
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 280, in get_result
INTERNALERROR> _reraise(*ex) # noqa
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 265, in __init__
INTERNALERROR> self.result = func()
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\runner.py", line 58, in pytest_runtest_protocol
INTERNALERROR> nodeid=item.nodeid, location=item.location,
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\terminal.py", line 219, in pytest_runtest_logstart
INTERNALERROR> self.write_ensure_prefix(line, "")
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\terminal.py", line 159, in write_ensure_prefix
INTERNALERROR> self._tw.write(prefix)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\py\_io\terminalwriter.py", line 207, in write
INTERNALERROR> write_out(self._file, markupmsg)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\py\_io\terminalwriter.py", line 342, in write_out
INTERNALERROR> fil.write(msg)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\colorama\ansitowin32.py", line 40, in write
INTERNALERROR> self.__convertor.write(text)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\colorama\ansitowin32.py", line 141, in write
INTERNALERROR> self.write_and_convert(text)
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\colorama\ansitowin32.py", line 169, in write_and_convert
INTERNALERROR> self.write_plain_text(text, cursor, len(text))
INTERNALERROR> File "c:\code\nexb\dev\deltacode\lib\site-packages\colorama\ansitowin32.py", line 174, in write_plain_text
INTERNALERROR> self.wrapped.write(text[start:end])
INTERNALERROR> IOError: [Errno 0] Error
Traceback (most recent call last):
File "c:\Python27\Lib\runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "c:\Python27\Lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "C:\code\nexb\dev\deltacode\Scripts\py.test.exe\__main__.py", line 9, in <module>
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\config.py", line 58, in main
return config.hook.pytest_cmdline_main(config=config)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
_MultiCall(methods, kwargs, hook.spec_opts).execute()
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
res = hook_impl.function(*args)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\main.py", line 134, in pytest_cmdline_main
return wrap_session(config, _main)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\main.py", line 128, in wrap_session
exitstatus=session.exitstatus)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
_MultiCall(methods, kwargs, hook.spec_opts).execute()
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
return _wrapped_call(hook_impl.function(*args), self.execute)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 250, in _wrapped_call
wrap_controller.send(call_outcome)
File "c:\code\nexb\dev\deltacode\lib\site-packages\_pytest\terminal.py", line 387, in pytest_sessionfinish
self._tw.line("")
File "c:\code\nexb\dev\deltacode\lib\site-packages\py\_io\terminalwriter.py", line 212, in line
self.write('\n')
File "c:\code\nexb\dev\deltacode\lib\site-packages\py\_io\terminalwriter.py", line 207, in write
write_out(self._file, markupmsg)
File "c:\code\nexb\dev\deltacode\lib\site-packages\py\_io\terminalwriter.py", line 342, in write_out
fil.write(msg)
File "c:\code\nexb\dev\deltacode\lib\site-packages\colorama\ansitowin32.py", line 40, in write
self.__convertor.write(text)
File "c:\code\nexb\dev\deltacode\lib\site-packages\colorama\ansitowin32.py", line 141, in write
self.write_and_convert(text)
File "c:\code\nexb\dev\deltacode\lib\site-packages\colorama\ansitowin32.py", line 169, in write_and_convert
self.write_plain_text(text, cursor, len(text))
File "c:\code\nexb\dev\deltacode\lib\site-packages\colorama\ansitowin32.py", line 174, in write_plain_text
self.wrapped.write(text[start:end])
IOError: [Errno 0] Error
(deltacode)
JMH@JMH-T510 MINGW64 C:/code/nexb/dev/deltacode (develop)
$
from deltacode.
from deltacode.
@majurg Yes. I first tested on Ubuntu and was able to run the test suite without error using the VSCode integrated terminal, as usual. I then tested on Windows 10 using Git Bash and that also has been working (so far) without error. So it looks like the problem is limited to the VSCode integrated terminal on Windows 10.
from deltacode.
@johnmhoran From looking at the error, it references ansitowin32.py
for colorama package, which deals with terminal colors etc. Not really sure how to solve it, as I do not use Windows or VScode.
It looks like you will have to use git bash or similar if you want a workaround on windows.
from deltacode.
Thanks @majurg . I agree. Not a problem. I was happy using Git Bash in tandem with VSCode before I started using VSCode's integrated terminal. If Git Bash fails, I'm also happy on Ubuntu 1604.
from deltacode.
@mjherzog Given your comment yesterday re build artifacts, would it be useful to include any additional file-related information, e.g., mime_type
, file_type
, programming_language
and/or is_binary
?
from deltacode.
merged #36, closing.
from deltacode.
Related Issues (20)
- Add limited delta stats after running HOT 1
- Upgrade DeltaCode to Python 3 HOT 4
- Create DeltaCode documentation on ReadTheDocs HOT 6
- Linux and MacOs buid is showing some warnings in TravisCI HOT 3
- configure failed,why? HOT 10
- Adding Azure Pipelines HOT 1
- In the output content format ‘[ ’how to understand? HOT 2
- Configure failed with "file setup.py not found" HOT 11
- Azure Piplines seems to be filing for Windows Test Jobs
- Create objects to score scan information HOT 6
- Add function to handle loading 2 codebases. HOT 2
- Remove redundant Scan model HOT 1
- Remove redundant File model
- Add Dockerfile
- Separate csv formatted output in Deltacode
- Merge DeltaCode in ScanCode TK
- Update structure to use the https://github.com/nexB/skeleton
- RFC: DeltaCode next! and roadmap HOT 2
- Update documentation after deltacode gets merge in scancode-toolkit HOT 2
- License detection diffs are incorrect HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deltacode.