Complexity: S

Complexity: S

pease review:
Wiki

• Try to find a definition of the ipython notebook files
• Try to find a library to parse these files?
• If not how difficult to parse yourself?
• Write up small independent example for parsing.

When I try to send a POST request using http://localhost:5000/api/projects/search or a GET http://localhost:5000/api/projects/dummyadd. I get the following error message from the browser (chrome):

post_request.html:14 XMLHttpRequest cannot load http://localhost:5000/api/projects/dummyadd. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.

(With postman it works)

I guess it has to do with CORS which has no impact on postman but on the normal browser. For testing you could use this file.

Check for frameworks that handle session management or even full user management via flask.

Create wiki page #7 with a basic API structure.
Update wikipage:

• projects (POST) @Krisselchen @DubbleClick
• projects (GET) @pdler
• projects/ (PUT) @Krisselchen
• projects/ (GET) @pdler
• projects/ (DELETE) @aTepliak
• index (GET)
• errorcodes
• add returns and link to corresponding formats or discribe them

Colour Codes:

Black: #333333
Orange: #FF5500

Image Type: PNG
Size: ~600px Width

It would be perfect if you directly upload (commit) the image into the folder web/style/img

Name: knex_logo.png

Due To: Thursday Night

Setup a docker file launching MongoDB and your python project.

Decide on which testing framework to use:

According to Flask's docs "Flask provides a way to test your application by exposing the Werkzeug test Client and handling the context locals for you. You can then use that with your favourite testing solution."
So we should be rather free in whatever framework we use.
The docs itself uses unittest as an example on how to do it.

unittest: unit test framework included in Python standard library, practical if users have experience with xUnit frameworks
Links: Introduction to unittest

Basic Example

pytest: not in standard library, more extensive test-discovery than the simple one unittest has, you need in general less code lines to write the tests, able to run unittest style tests x
Links: Introduction to pytest

A third rather popular option seemed to be nose, however that one is not further developed and from what I've read most people switched to pytest, and some to unittest.

In the end both seem to be of use, they have different advantages, though from what I've seen pytest seems to be the most popular, and it was actually recommended to me by a friend who has already worked with flask together with pytest.

If anyone else has any recommendations or has already any experience with any of these (or something else) please feel free to share

Ideas

• Roboto
  https://fonts.google.com/specimen/Roboto
• Advent Pro
  https://fonts.google.com/specimen/Advent+Pro
• Open Sans
  https://fonts.google.com/specimen/Open+Sans

Complexity: XS ,-)

{"_index": "projects-index", "_type": "Project", "_id": "1", "_version": 112, "result": "updated", "_shards": {"total": 2, "successful": 1, "failed": 0}, "created": false}{"_index": "projects-index", "_type": "Project", "_id": "2", "_version": 112, "result": "updated", "_shards": {"total": 2, "successful": 1, "failed": 0}, "created": false}{"_index": "projects-index", "_type": "Project", "_id": "3", "_version": 112, "result": "updated", "_shards": {"total": 2, "successful": 1, "failed": 0}, "created": false}

is not valid.

• Evaluate Libraries for PDF parsing
• Maybe writeup small (backend independent) example, if it is easy to do.

• Check how to export shared file (without credentials, where possible) as something parsable (txt? rtf? maybe pdf #62 )

Complexity: M

If you click the name of the project in the table of ProjectDiscovery, you should land on the Project Details Page with the related Project Information.

Complexity: S

We don't wanna filter by Description right now, so remove also the input field for it.
I'm gonna update the mockups.

• Move app.route functions into projects.py in a new api folder (see structure below)

• Check that it is still working and fix if needed

• Rename the flask_api.py into init.py (see structure below)

• Make sure everything works again

The overall structure of the project should look like this:

• backend
  • init.py
  • api
    • projects.py
    • (later users.py, etc)

Complexity M
Search Query

Complexity: M

Writeup coding conventions (one done #3), explain overall structure, etc.

• React Tables
• Pagination
• Autocompletion

Complexity: S

Right now we use some online resources. to guarantee that the application in an offline environment works as expected we have to import every dependency locally.

• fonts
• frameworks
• glyphicons
• images

Complexity: S

Task One: (Priority - High)

• We want that the simple search actually filters by name in the project discovery.
• We also want the filters can be combined in detail search.
  The results of the filtered data should be displayed in the table underneath the search.

Task Two: (Priority - Low)

• We want to sort tables by name (A-Z, Z-A), status (done, in progress, pending), by date (newest - oldest, oldest - newest)

For the beginning I recommend that we filter client-side.

What?

Upload some testdata to the database automatically for test framework usage

Tasks:

• Create a new "tests" folder, populate it with some random data. (Ask Marco)
• Create a new testing.py (possibly initializing the test framework) and read the test data, upload it via http client to the flask server (see API - POST /api/projects).
• Create new docker-compose-test.yml that runs the tests as new python container
• Let travis.yml run the docker-compose-test.yml file
• Write integration tests for the existing api endpoint (see wiki page)

Research : Syncing MongoDB and Elasticsearch

I've looked into tools for syncing, and these were the mentioned options:

mongo-connector

creates a pipeline from a MongoDB cluster to one or more target systems, like Elasticsearch. It synchronizes data in MongoDB to the target then tails the MongoDB oplog, keeping up with operations in MongoDB in real-time. Can be easily installed with pip and seems to be the popular option since mongodb river is deprecated (for river read further).
See also the wiki of mongo-connector: Usage with Elasticsearch

Transporter

is for synchronizing data e.g. from mongoDB to elasticsearch, however, that does come as a one time export and would need to be modified to keep it in sync (feature only in beta).

logstash

Logstash is an open source data collection engine with real-time pipelining capabilities. Logstash can dynamically unify data from disparate sources and normalize the data into destinations of your choice.
Has a plugin for output to elasticsearch, but only a community-maintained non-verified plugin for mongodb input. Probably possible but doesn’t seem like the easiest option.

MongoDB river seems to have been a popular choice, as well. But that one is deprecated and won’t work with the newest MongoDB version.

Conclusion

In the end I’d personally stick to mongo-connector, which had been suggested at first, too. Easily set up and actually made exactly for our purpose.

Task One: (Priority - High)

So, this ticket is about getting actual data from backend and displaying them in the Discovery View.
Therefore we need an API call that returns the manifest.json. The results should be shown in the table underneath.

A table row contains information about:

• Project
• Name
• Author
• Status
• Description
• Date

Task Two: (Priority - Medium)

We also wanna include a pagination that allows a user to switch between the result pages.
The maximum amount of table entries per page should be 15.

The post method has to handle multiple projects in one manifest

Complexity: L

I guess it would be the best to split this ticket so that two people can work on it.
One should work on JSON, the other one on "Create by Template".

But this is only a suggestion.

Update: We only need "create by JSON" as "create by template" is already done.

• analysis now array
• array on top
• make both possible array and object on top
• allow url to be (optionally? if easy) to be an array
• Make nice template

Complexity: M

This should include fetching only 15 elements/projects per page! We can adjust the amount of projects later on.

Basic travis support:

• Runs tests
• Checks coding conventions
• Builds docker maybe?

Complexity: M

A user should be able to edit existing projects. This happens via a button on the ProjectDetail Page. After the User presses it, he should see the CreateProject by Pattern View with filled Project information. Here can he edit the Project and save the changes by a button. For clarification I'm gonna attach a mockup (but this is not really necessary, you can already start without it)

last week we've created the most important views, for the next step it's needed to transform them into accessible pages where we can actually navigate through.

Complexity: M

I have to create Mocks first and think about possible cases. I'll keep you updated.

Create some folders.

Maybe

• resources
• frontend
  • .gitignore
• backend
  • .gitignore
• README.md
• .gitignore
• bla bla bla

Create some example json files (not final format) and start implementing a basic validator.

Agree on coding conventions.

Tabs: 4 Spaces
Format: UTF-8
Line breaks: Unix Style

Recommendations:

• Python: docstrings, pep8 with increased line limit 100
• Javascript/JSX: no idea