CINES project repository
nssac / cines Goto Github PK
View Code? Open in Web Editor NEWCINES project repository
License: Other
CINES project repository
License: Other
landing_page_material_feb_20_2020.pdf
We modified the landing page document. In particular, we removed Persistent and New City from the list of "Participating Institutions and Organizations". A new version of the landing page document is attached.
We have met with NetworkX. Not much can be done. They have no resources to change source code formatting so that we can auto-parse the source code comments for each method and thereafter create a main() method [driver code] for each method.
So, make up an alternate plan.
Find discussion on the approach for allowing users to submit queries. We spoke about providing building blocks for building queries so that we can constrain what they can submit as a query.
TODO: find that info.
Dustin to move pages up there for access.
The Snap methods are listed in the github repo here:
https://github.com/NSSAC/cines-snap-texts/tree/master/methods-to-implement
Kuhlman has to go through again and make pass to:
-order them.
-put in major sections.
-put in changes to inputs that are required. This includes changes to the Snap text files that are parsed, OTHER????
This is to get a feel for the types of cases and for what the main() method must look like. The main() method is our target.
This method is under development. So we cannot develop a driver main() method for it.
Hence, in the file in this github repo: https://github.com/NSSAC/cines-snap-texts/tree/master/methods-to-implement,
put under the method name, the following headings:
number: XX (put in a number)
alter Snap input file: this method cannot be run. The input file says that the method is under development.
parse: cannot be done.
driver generated: cannot be done.
driver executes: cannot be done.
driver verified: cannot be done.
Kuhlman can grab about 10 networks that he has laying around and put them into sciduct, for demostrating operations in sciduct.
The use case for this would be to run web site scrapers to generate data (that can be used to make networks), periodically. There are probably other such cases. So this is a general sciduct question (versus running a task when specifically requested).
All arguments in snap are listed as either input or output. But there are Snap methods that are labeled input, while actually being inout. So put that feature into parser code and driver constructor code.
Since we cannot
This is the design of the web pages for the web app, for:
-displaying content
-moving among pages
-operations users can perform on a page
-queries
-subqueries
-outputs beyond screen displays (like writing results of a query to file).
Simplified version of architecture.
Prepare for meeting with Naren. Talk with him about:
-plan
-timeline
-KUHLMAN: check about sharing student between Naren and Cliff.
Plan
-scraping: identify types of networks.
-generate code to scrape each type of data sets. We want to have them scrape now, and run these codes periodically to scrape going forward.
-stand alone codes to generate graphs of various types from the scraped data.
Naren's task is to generate networks by scraping data. We need to establish scope. For example, do we want him to just scrape, and then we make the networks, or do we want him to scape data and make networks. If we have Naren do the former, then we can get more scraping from them.
I have added a new version of the material for the landing page. (See the attached file below.)
The changes in this version are as follows.
(a) The URL for Network Repository has been changed from "networkrepository.org" to
"networkrepository.com". (This correction was provided by Ryan Rossi after the Kickoff meeting for CINES on Feb. 28, 2020.)
(b) At the suggestion of one of the participants in the Kickoff meeting, the description of each link under the "Links to Educational Materials on Network Science" has been slightly reorganized. In particular, each link description specifies the name a person (or an orgranization), the person's affiliation and location, name of the course (or book or topic) followed by the URL.
(c) A new link to the Network Science class taught by Natarajan Megahanathan at Jackson State University during Fall 2019 has been added.
Final note: We are collecting the logos for the various partners. (Chris Kuhlman has set up a github repository for the logos. Each logo is either a png or a jpg file. If Persistent needs these in some specific format, please let us know.
As usual, if you have any questions, please let me know. Thanks.
-- Ravi
We need graphics and sexiness for the home page of the n.s web app.
There are ideas on the google doc here:
https://drive.google.com/drive/folders/1IsALmpBRLb9ebv8mSxPqEkA4tlwB5qau
Currently, there is one return object in the intermediate representation (IR). There needs to be two: one for the library method, one for the main() method.
The Snap text file GenBaraHierar.txt contains about five lines that show a graph.
This is a highly non-standard format for the input file that describes the Snap method GenBaraHierar().
Therefore, delete these lines of text, and then operate on the resulting file.
This library method is covered in the file in the github directory here: https://github.com/NSSAC/cines-snap-texts/blob/master/methods-to-implement/library-methods-to-implement.md
And it says for this method to delete the five or so lines that contain the graphic.
Make a PPT presentation to share with, and go over with, NetworkX folks.
Integrate the SAT solver, Glucose, to sciduct.
This is a C++ stand-alone code.
This code is being prepared by Josh, with comments and description.
We need to integrate into sciduct.
See this page is an example of a layout.
I have attached a new version of the maerial for the landing page. (A minor revision was done on Feb. 19, 2020.) I have given below the changes in the new version compared to the previous one.
(a) The material on "Participating Institutions and Organizations" has changed. In the current version, we have the name of the institution (or organization) followed by one or more contacts for the entity.
(b) Names and affiliations of the members of the Scientific Advisory Board have been added to the document.
(c) The part on "Links to Educational Materials on Network Science" has some additions, namely the links to the courses taught by Aaron Clauset, David Kempe, people at Stanford University and Boleslaw Szymanski. Also, please note that for the courses taught by Reka Albert and Albert-Laszlo Barabasi, the correct language accents have been added to their names.
If you have any questions, please contact me. -- Ravi
This meeting took place on 06 May 2020 among
-Serban Porumbescu (of Gunrock)
-Dustin Machi
-Lucas Machi
-SS Ravi
-Chris Kuhlman
This is a series of ~4 meetings to go over various aspects of work:
-overall goal and approach.
-code walkthroughs of two codes.
-go over multiple examples.
-go over scripts to run codes, compare new and verified output, run regression tests.
We need to develop a plan for integration of Gunrock into net.science.
Currently, the elements are:
-download code.
-they have nice documentation in Doxygen.
-put on rivanna.
-work out (play) with code for executing on rivanna GPUs.
-main() method driver codes are already provided by Gunrock.
-run each code and generate a couple of verification cases per method, for testing purposes.
-generate (auto-generate) the sciduct input and output schema files and the job definition file (and others associated with sciduct).
-run test cases on rivanna prior to integration into sciduct.
-integrate codes.
-rerun test cases.
-right now, we go with C++ code; Gunrock's python wrappers are under development.
-github site exists for Gunrock.
-Gunrock has test cases for their GPU-target codes and for CPU-target codes (for verification). (/gunrock/apps/hello, for example).
-We can provide them value by providing singularity container of their code.
We need to discuss this. These steps were put in based on a meeting with Gunrock on 06 May 2020.
Chris, did you get a notification of this issue?
These are slides to describe system.
Network Repository has many networks.
We need to connect to NR to retrieve graphs and store in scidut
This is part of the pipeline to auto-generate, for each Snap library method, main() methods and integrate them into sciduct. Here, we need to auto-generate the input files to sciduct, to run a main() method within sciduct.
I have attached a pdf of the revised version of the landing page material for CINES.
In this version, I have included some publications that acknowledge support from the
NSF Grant for CINES. (I have a red comment to indicate the new part.)
I have listed all the papers where I am a coauthor. It will be helpful if
Chris and Dustin can take a look and add any publications that I may have missed.
Thanks.
These are older networks. But this is a first step to establishing a pipeline in moving graph data from PATRIC to net.science.
We manually copy graphs from PATRIC and put in net.science.
Manually make main() methods for a selected number of NetworkX functions so that we know what our goal is, i.e., what the output from this work must be---the content of the main methods.
Another issue addresses the driver constructor code for NetworkX.
This issue is to address the preceding step: parsing code or comment files to generate intermediate representations (IRs), as follows.
Write a code to do the following:
-parse a NetworkX method in a source code file.
-use only the method signature.
-put all info from the method signature (method name, argument name, and return variable and/or type) into an IR.
-this not only enables us to fill in particular fields, like name of arguments in signature, but it enables us to identify other fields that need filling in. For example, given a variable name in a signature, we need the variable type, the signature variable type (i.e., input, output, inout), descriptions, etc.
-for all fields whose entries are not known, write something like "PLEASE FILL IN".
This way, if we have we produce more than a skeleton: we fill in what we know and the person who manually fills in the IR knows what fields need to be populated.
The goal here to create a channel with PATRIC and pull network data from PATRIC periodically, to based on some event.
Data are brought over from PATRIC and graphs are generated within net.science.
Make slides on the solution approach and go through with Dustin, Lucas, Ravi.
This is the approach to auto-generating a main() method for each Snap library method.
The NetworkX code is not in a shape that we can auto-parse the comments for method source code, and then create driver methods as we are doing for Snap. Therefore, we want to meet with NetworkX folks and determine whether there are other solutions.
Dustin to work with Lady from rivanna whose name I cannot recall. Michelle.
Prepare to meet with Ed Fox.
-What do we want them to do?
-Do we want to have framework, code based into which their work goes?
-Types of DOs.
-What is the split between Fox and Persistent.
Go over listing of methods here in github:
https://github.com/NSSAC/cines-snap-texts/tree/master/methods-to-implement
Make second pass through this file to organize, prioritize, add comments to help persistent with integration.
This method is under development. So we cannot develop a driver main() method for it.
Hence, in the file in this github repo: https://github.com/NSSAC/cines-snap-texts/tree/master/methods-to-implement,
put under the method name, the following headings:
number: XX (put in a number)
alter Snap input file: this method cannot be run. The input file says that the method is under development.
parse: cannot be done.
driver generated: cannot be done.
driver executes: cannot be done.
driver verified: cannot be done.
Meet with NetworkX folks, led by Aric Hagberg. The topic is to see what can be done to unify formatting, and make code commenting complete to a degree that we can parse. it.
This is a "catch-all" issue for Persistent's work with generating and verifying Snap method driver codes.
Verify (again) the first 30 cases of creating SNAP library method main() methods. Codes and data exist, and output exist. Need to verify.
Hi Chris and Dustin,
A new collection of graph data sets for research on machine learning
has recently been released. It is called "Open Graph Benchmark" and its
description appears in
https://arxiv.org/pdf/2005.00687.pdf
It appears this data set was created jointly by Stanford, Harvard,
Dortmund University in Germany and Microsoft Research. We should consider having this data set as part of net.science. Thanks.
Make plan to integrate this work.
Integrate the SAT solver, Clasp (https://potassco.org/clasp/), to sciduct.
This is a stand-alone code that is wrapped in python.
This code is being prepared by Josh, with comments and description.
We need to integrate into sciduct.
April_16_2020_landing_page_material.pdf
I have attached a revised version of the document containing material for the landing page of the website for CINES. To make it easier to see the new material that I have added, I have tagged all the new material using the word "(New)" in red. If you need any clarifications, please feel free to contact me. Thanks.
This is content for landing page.
The approach is to put small graphics and a little text, for each of several topics or sections.
Candidate sections are in this google document here: https://drive.google.com/drive/folders/1IsALmpBRLb9ebv8mSxPqEkA4tlwB5qau
Look over NetworkX source code, and particularly for the commenting of the library methods (entrypoints). Determine how uniform, complete these comments are, for parsing.
Plan for the code that generates the main() method per library method.
A SAT solver to be commented and delivered by Henry and Ravi.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.