thiennc1107 / search-analyzer

Search analyzer

An web application to analyze search results of various keywords.

Core feature

• Dashboard the view analytic results of each keyword search results.
• File upload page to upload the list of keyword to analyze.
• Basic authentication

Technologies used

Backend

• Go programming language: A highly scalable but also productive programming language.
• Echo: A high performant http library for go.
• Ent: A Entity like ORM for go which is supported by Meta

Frontend

• Htmx: high power tools to make HATEOAS application without compromising interactivity from SPA framworks like React, Vue,...
• Bulma CSS: free, open source framework that provides ready-to-use frontend components that you can easily combine to build responsive web interfaces.
• Alpine JS: Minimal and lightweight frontend framework.

Getting started

Setup

• After checking out the repository, from within the root, start the Docker containers for the database and cache by executing make up:

git clone [email protected]:thiennc1107/search-analyzer.git
cd search-analyzer
make up

• After setup the environment, you can start the application by executing make run, by default, the application should be able to access in browser at localhost:8000. The database scheme will also be automatically migrated.
• If you want to reset all data, execute make reset

Configuration

The config package provides a flexible, extensible way to store all configuration for the application. Configuration is added to the Container as a Service, making it accessible across most of the application.

Be sure to review and adjust all of the default configuration values provided in config/config.yaml.

search-analyzer's Issues

Lack of Git flow

Issue

Even though you worked solo in this project, it was a clear requirement for this code challenge that:

Use Git during the development process. Push to a public repository on Github or Gitlab. Make regular commits and merge code using pull requests.

We expect the changes to be merged by using pull requests; the reason is it helps show your planning skills, i.e. how you analyze the given requirements, break it down into smaller achievable parts, and implement them in each pull request. Also, by going through the PRs, we can evaluate if you know to use Git flow efficiently (which is crucial when working in a development team)

Start of the code review process 👋

Hello Thien 👋 ,
Thank you for your effort on the code submission. I am Long, engineering lead at Nimble, and I am happy to be the reviewer for our code review session. First, I am sorry for the late response as I was on leave.

During the review process, I would like to know more about your decisions, and thus I will create issues where I think there could be more improvements regarding your performance here.

At the same time, please keep in mind that this is a bi-directional process, and I would love to hear back from you as well. Therefore, do not hesitate to raise your questions or share your opinions about the implementation (if any) during the process.

If we are aligned on any issue, and you would like to correct them, please address the issue using a proper git flow (creating a new branch, opening a Pull Request (PR) per issue, and merging the code when you are ready), and I will follow up on those fixes. Please note that you don't have to close any of my created issues after merging your PRs, I will help verify and close them for you once they pass. 😇

We expect the code review process to be completed within 2-3 days at most. As a result, ensure you are responsive during this process. If you need additional time, please inform us right away so that we can plan accordingly.

In the end, I really do hope that you find the process enjoyable. Good luck and happy coding. 🤘

[Question] What is the plan to workaround Google rate limiting

I'm aware that the part about scraping the search results from Google has not yet been implemented. However, if you had more time, what would be your solution to do so, and how would you will work around Google rate limiting?

I'm also aware that you raised a concern previously via email

I realize the Google search result has changed a lot since last time I crawled, which I believe makes the challenge much more difficult than it used to be :-(. I don't know if the one who gave the challenge is aware of this change yet.

Since you didn't provide more details, it is hard to know which issue you faced, but we didn't notice significant changes that prevented a possible solution to extract the search result.

[Suggestion] There should be a setup instruction

Issue

Given that not everybody will be familiar with the technical stack chosen for any given project, it is always recommended to have a setup instruction to document the process to set up and run the application on a development (and other) environment(s).
This application provides makefile with various commands + docker-compose file so that some simple guidelines would be much more helpful.

I managed to set up and run the application on my local, so fixing this issue is unnecessary. This is just a suggestion :)