muly / product-track Goto Github PK

need to troubleshoot and fix . run make test to reproduce this

--- Failed steps:

  Scenario: test product availability # integration_testing/features/web-scraping.feature:2
    Then the response should be "amazon_available_product_response.json" # integration_testing/features/web-scraping.feature:8
      Error: {Url:http://localhost:8006/mock/amazon_available.html Price:49.99 Availability:true Image: Name:} is not equal to {Url: Price:0 Availability:false Image: Name:} 

  Scenario: test product availability # integration_testing/features/web-scraping.feature:2
    Then the response should be "amazon_unavailable_product_response.json" # integration_testing/features/web-scraping.feature:8
      Error: {Url:http://localhost:8006/mock/amazon_unavailable.html Price:0 Availability:false Image: Name:} is not equal to {Url: Price:0 Availability:false Image: Name:} 

  Scenario: test product availability # integration_testing/features/web-scraping.feature:2
    Then the response should be "flipkart_available_product_response.json" # integration_testing/features/web-scraping.feature:8
      Error: {Url:http://localhost:8006/mock/flipkart_available.html Price:123 Availability:true Image: Name:} is not equal to {Url: Price:0 Availability:false Image: Name:} 

  Scenario: test product availability # integration_testing/features/web-scraping.feature:2
    Then the response should be "flipkart_unavailable_product_response.json" # integration_testing/features/web-scraping.feature:8
      Error: {Url:http://localhost:8006/mock/flipkart_unavailable.html Price:0 Availability:false Image: Name:} is not equal to {Url: Price:0 Availability:false Image: Name:} 

  Scenario: test product availability # integration_testing/features/web-scraping.feature:2
    Then the response code should be 406 # integration_testing/features/web-scraping.feature:9
      Error: 405 is not equal to 406

current message:

is the activeTab permission not good enough? do we also need the tabs permission?

this will be backup option to use when the scraping fails to get the data

tables:
track_request

• get using filter
• get by id
• delete by id
• create/insert
• update by id

when user click the extension, a popup should be shown with below options

• track availability
• track price (with a text box for min price threshold)

and a submit button.
when submitted, the respective API should be called depending on if the user selected the track availability option or track price

using cucumber/karate tests

• add support for service mock web pages
• mock web pages: amazon product web page for available product (price and quantity)
• mock web pages: amazon product web page for unavailable product
• mock web pages: flipkart product web page for available product
• mock web pages: flipkart product web page for unavailable product
• integration testing framework: simple feature file with support to run test from go test
• improve integration testing in Data Driven Testing format
• update feature data table to cover the below scenarios:
• - amazon available product: verify the price and availability
• - amazon unavailable product: verify the unavailability
• - flipcart available product: verify the price and availability
• - flipcart unavailable product: verify the unavailability
• - unsupported website

• add Google logo.
• use the google recommended button if possible.

monthly email with the table of unsupported website host, followed by count of number of requests attempted, in the decreasing order of the count.

create a go project template repo based of github.com/muly/product-track repo to be used to create new gcp app engine based golang application from this template using gonew command.

resources to explore:

• https://go.dev/blog/gonew
• https://cloud.google.com/blog/products/application-development/simplifying-creation-of-go-applications-on-google-cloud
• https://www.youtube.com/watch?v=EbQZhMHv9oU
• https://www.youtube.com/watch?v=HJHCndEVoiA

Version: 1.1.1
Status: Rejection
Violation date: Dec 21, 2023

Violation type: Spam and Placement in the Store
Details:

• Violation reference ID: Yellow Magnesium
• Violation: Not providing promised functionality
  • ’Invalid value for ‘oauth2.client_id’
• How to rectify: Test the code that you submit to the web store locally, verify that your submission contains the files you expect at the paths you expect, and is working as intended. For more information, please refer to the documentation and migration checklist on Manifest V3 version.
• Relevant section of the program policy: Extensions with broken functionality - such as dead sites or non-functioning features - are not allowed.

Acceptance criteria

• able to run the api local and test using postman or curl
• user endpoints:
  • [POST] availability track request from user /track/availability
  • [POST] price track request from user /track/price
• internal endpoints:
  • [POST] availability and price of a product /product

[POST] /track/availability:
input:

• url string

output: http status only

[POST] /track/price
input:

• url string
• min threshold price float

output: http status only

[POST] /product
input:

• url string

output:

• url
• price
• availability

router library: julienschmidt/httprouter

this is to avoid sending repeated notification whenever the cron job run.

cron job: https://console.cloud.google.com/cloudscheduler?project=smuly-test-ground

• frequency: once a day

new endpoint: [POST] /execute-request

• goes through all the entries in the track_requests table and executes the track availability or track price requests and triggers the notification (not in the scope of this task, for now just call a dummy send notification function that just logs)

some pointers:

• how user data is collection:
• what is done with the user data collected:
• any trackers (like cookies) are used? No
• any ad networks used? No
• how to opt in to this data collection? by installing this chrome extension
• how to opt out of this data collection? ?????
• how user can request to delete their data or download their data?
• contact information for app owner?

example url: https://www.azurestandard.com/shop/product/food/grains/wheat/hard-red-wheat-berries/hard-red-wheat-organic/11656?package=GR062

current: our extension doesnot support the given url

new:
Product Tracking app doesn't (yet) support tracking products on www.amazon.com
Product Tracking app doesn't (yet) support tracking products on www.azurestandard.com

design: TBD

design questions:
to which email address the notification to be sent?
where to save this email address in the database?

scenario:

1. user searches for google pixel and select the google pixel 7a 5g and request the track request.
   2, and later same user searches for google pixel 7a 5g and select the google pixel 7a 5g and request the track request.

the url for these two instances will be different even though the product is same.

for example:
https://www.amazon.in/Pixel-Sea-8GB-128GB-Storage/dp/B0CMHLNVGS/ref=sr_1_4?crid=29M7LBVDXVQCP&keywords=google+pixel+7a+5g&qid=1700846878&sprefix=google+pixel+7a+5g%2Caps%2C159&sr=8-4
https://www.amazon.in/Pixel-Sea-8GB-128GB-Storage/dp/B0CMHLNVGS/ref=sr_1_3?crid=15JYE7A72BD2R&keywords=google+pixel&qid=1700846833&sprefix=google+pixel%2Caps%2C187&sr=8-3

above two url are for the same product, but because user visited this page using different search keywords, the url formed is slightly different. because of this, when user tracks again, a different track request is created, which it should not.

one solution: strip the use to remove the extra information before storing in our database. for example only store the below as url.
https://www.amazon.in/Pixel-Sea-8GB-128GB-Storage/dp/B0CMHLNVGS/

example: if scraping fails, use the supported api function to get the data.

might need changes to product-scrape repo(like returning multiple functions instead of only scraping function). need to think and come up with the design, and implementation.

• well documented deployment steps in readme
• makefile command support

• find next 10 popular ecommerce websites
• inspect and find the tags for scraping logic
• are these websites fitting in the template of the generic scrape() function that we have?
• save the scrape tags metadata into a private website
• modify the code to import this website to scrape.

• move the unit test functions into separate test files. ex functions from units.go should have the unit tests in utils_test.go
• remove fmt.Print* and replace with log package
• add make file: build, test

manifest.json:

• permissions:
• host_permissions

right now, it is not clear from the chrome extension icon if it is local deployment or installation from chrome web store. we need to have a different logo for local installation of chrome extension.

• new logo with simple text such as PT
• make file and script update to use the new logo file when preparing for the chrome local deployment.

• collect zip code from user and save to db (track_request table)
• use the zip code for the availability logic.
• zipcode in the ui should only showup for specific website that needs it to find the availability, not for all the websites.

add git commit info in logs:

• latest commit hash
• if there are any uncommitted changes at the time of deployment: yes/no

send a rich html email instead of plain text

it will be valuable to know this information to plan supporting new websites based on the demand.

changes:

1. when the track request validation error is websiteNotSupported, before returning with status code StatusNotAcceptable, we should save the hostname in a new table. this table should have the unsupported host and the timestamp of the request. later we can use this information to count the number of requests coming for each of those unsupported websites and then plan tto support the recent ones in high demand.

2. we should also update the chrome extension to send better message (on StatusNotAcceptable status code is received) to hint the user, that we know that the user wants to track product from that host and we will try to include support for it.

design: TBD

ref: https://stackoverflow.com/questions/51872972/automate-github-commit-to-google-appengine-version

user authentication
add user id to track request ids