Comments (13)
@johnmhoran Plan of attack:
- Review + merge PR #20
- Modify Delta object
to_dict
function to return only minimal info (category and path for the most part) - Modify CSV ouput function to handle new data structure
- Modify JSON output to Add header information that has been removed from
DeltaCode.to_dict()
from deltacode.
Feel free to push your changes in a branch without opening a PR. This is easier and less noisy than constantly updating a PR.
If you have questions or would like me to expand in detail on any of the above items, let me know.
from deltacode.
@majurg I've modified Delta.to-dict()
and the CSV output to handle the new data structure, and fixed the 9 failing tests as well. About to add missing headers to the JSON output. Here's an excerpt from the current JSON output file testing a set of test scans with 1 added file:
{
"added": [
{
"category": "added",
"path": "a/a5.py"
}
],
"removed": [],
"modified": [],
"unmodified": [
{ . . .
How do we want to handle the redundant category information? Replace the 4 category keys with a single key named deltas
and a value containing a list of category
/path
key/value pairs? Example:
{
"deltas": [
{
"category": "added",
"path": "a/a5.py"
},
{
"category": "unmodified",
"path": "a/a1.py"
} . . .
Also, I assume we want the missing header info added to the top of the JSON output file. If so, I think that means adding values to the top of the incoming OrderedDict created by DeltaCode.to_dict()
. Do I have that right??
from deltacode.
@majurg I just committed and pushed my work to date so you can vet including in connection with my recent questions.
from deltacode.
We can keep the redundant categories around for now. If you recall our conversation yesterday, the delta object's category field will no longer match the Deltas dict category keys once we add license/copyright information.
from deltacode.
I do recall, e.g., license_change
. Thanks @majurg . When you have a chance, can you sketch out an example excerpt of how you see the future JSON structure?
Re the 2nd question, do we want to add the missing header info to the top of the incoming OrderedDict. If yes, how does one do that? I've seen 2 approaches: rewrite the OrderedDict -- said to be slow but relatively straightforward -- or write a function to prepend.
Finally, once I finish this, what shall I tackle next?
from deltacode.
@majurg I can add the version header from inside the JSON function by moving the variable from __init__.py
to cli.py
, but adding the stats from DeltaCode.get_stats()
has eluded me so far. Do we need to restore that to the DeltaCode.to_dict()
method in order to add it to the JSON output file?
from deltacode.
@majurg I'm able to add the stats by passing new
and old
to generate_json
and calling the stats like this: data['deltacode_stats'] = DeltaCode(new, old).get_stats()
. The 2 headers (version and stats) have been added to the JSON file, though they appear at the bottom rather than the top (per my question above re adding to the top).
from deltacode.
a = OrderedDict([
('header', header_info),
('deltacode_stats' deltacode.get_stats()),
('deltas', deltacode.to_dict())
])
pass the deltacode object into generate_json
instead of the data dict and just call to_dict inside the csv function at the right place.
from deltacode.
Shouldnt need to move version to cli.py. Just import deltacode
at the top and reference the version like: deltacode.__version__
from deltacode.
Very nice. Thanks, @majurg .
My only open issue: the list comprehension you suggested -- still digging into how to include the 3 variable assignments and the tuple_list.append()
operation currently inside the 2nd of the nested for
loops.
from deltacode.
@majurg I believe I've answered my last question. Spent time yesterday trying to figure out how to address the double-nested multiple variable assignments and the append
operation inside a list comprehension. Nothing seemed to work, no hints from my research.
Took a fresh look this morning and realized that I've seen this before in simpler form: initializing variables earlier than necessary. When all is said and done, it's just an append
operation.
And this:
tuple = ()
tuple_list = []
deltas = data
for delta in deltas:
category = delta
for f in deltas[delta]:
category = f['category']
path = f['path']
tuple = (category, path)
tuple_list.append(tuple)
. . . can be replaced with this:
deltas = data
tuple_list = [(f['category'], f['path']) for delta in deltas for f in deltas[delta]]
All 79 tests pass. I want to do a little command-line testing just to be sure, and if all looks good, will clean up the code, commit, push and open a PR.
from deltacode.
#21 merged, closing this.
from deltacode.
Related Issues (20)
- Add limited delta stats after running HOT 1
- Upgrade DeltaCode to Python 3 HOT 4
- Create DeltaCode documentation on ReadTheDocs HOT 6
- Linux and MacOs buid is showing some warnings in TravisCI HOT 3
- configure failed,why? HOT 10
- Adding Azure Pipelines HOT 1
- In the output content format ‘[ ’how to understand? HOT 2
- Configure failed with "file setup.py not found" HOT 11
- Azure Piplines seems to be filing for Windows Test Jobs
- Create objects to score scan information HOT 6
- Add function to handle loading 2 codebases. HOT 2
- Remove redundant Scan model HOT 1
- Remove redundant File model
- Add Dockerfile
- Separate csv formatted output in Deltacode
- Merge DeltaCode in ScanCode TK
- Update structure to use the https://github.com/nexB/skeleton
- RFC: DeltaCode next! and roadmap HOT 2
- Update documentation after deltacode gets merge in scancode-toolkit HOT 2
- License detection diffs are incorrect HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deltacode.