airavata-courses / garuda Goto Github PK

Date is not changing on click

Populate reflectivity in DB

Possible solutions:

1. Sampling the dataset
2. Compressing the dataset while storing and retrieving data.
3. Retrieve data by parts

Need to add few more parameter from queue_worker (start_time, end_time, requestID)
Tested rest of the parameters

Originally posted by @pranavacharya in #33 (comment)

Just ran your project, and I have a couple of comments and suggestions -

Build time took around 12-14 minutes, was easy to follow from the readme files.

1. Implementation of CircleCI for continous builds on commits
2. Using docker-hub instead of individually building each microservice in your main repo
3. Time slot selection doesn't seem so intuitive at all
4. Could've explored more properties instead of just reflectivity
5. Also not sure how you're handling the error when there's no data for the querry

Rest great work! Loved your project as such

Reviewed by Team Scapsulators (https://github.com/airavata-courses/scapsulators)

data_extractor multi thread downloading
added feature to not download if existing

• consume offload request from rabbitmq queue
• extract data using data_extractor
• publish message / call db writer service

Add CI using github actions for data_extractor module on data_extractor and main branch

Refresh page when a new request is submitted

Docker compose for whole project

add response from db_middleware in 'data_dump' key

Add remote repository setup for docker images

When the same request is made multiple times, nothing happens on the UI.

I would like to point out that the weather data on the map can be made more informative by providing some visual aids like a heat map or reflectivity index.

Likes:

• System architecture looks very similar to the way we thought.

Observations:

• Each microservice needs to make an API call to the DB middleware microservice to make DB operations. Here, it increases one network hop which might hamper performance.

Piyush Nalawade
Team Zilean 🦄

Show all user requests in a list

removed last plotted map from view

added test for db_middleware ping path

Python microservice hosted using Flask to get queries from the dashboard and return data to be shown.

Congratulations on coming up with a smooth UI and data flow.

UI seems very minimal and intuitive.
Below is the resultant data:-

There are some points I will which we want to bring to your knowledge:-

• You tried well to persist the data to db for any search before returning it to the user. But It seems some of the time durations have huge data that it's not getting retrieved properly because of the MongoDB limitations. Maybe you can compress the lat-long data to already compressed data and then store to DB. So that merging documents won't reach the limit.
• Docker Images can be reduced in size using alpine os images of Node, python and Java(Not sure about this)
• Build time can be removed by pushing images on Docker Hub

Review Done by Team Neo(https://github.com/airavata-courses/neo)

Add docs

added CI for queue_worker

Moved data_extractor CI to garuda__github_actions_CI

Unit test for data_extractor project

modify data extraction module to change data format, ie. azimuth distance to lat,lng

Hi Team,

I tried to set up your Project on my System which is a MAC running (Monterey v12.1) in which port 5000 is being already used by the OS and I saw that your api gateway runs on PORT 5000, so I tried to update it to an open port in my system (Port 6000) but it still didn't work and then I found out that you have written the port in files other than the configuration files and even after changing them it didn't work.

I would suggest not to write PORT details directly in the code like it was done in garuda.apigateway/server.py and you can also try to look what can be done so that such issues with PORTS can be handled with less changes or changes at a single place like changing the port in docker-compose.yml maybe should change the PORT dependency everywhere.
You can have a look at "Environment" tag for docker-compose.

React webapp setup and create login page

create deployments and statefulset with auto scaling for all micro services

Containerization with dynamic hostnames

A feature to move all the hostname and port to env variable and make appropriate changes in docker files,

db_middleware feature to add support for 16mb document