relatable-course-visualization / relatable

Relatable is an interactive course detailer for the University of Saskatchewan. A user searches for a course, and the application displays its prerequisites, dependencies, and other useful information.

Product Description

Relatable is an interactive course detailer for the University of Saskatchewan. A user searches for a course, and the application displays its prerequisites, dependencies, and other useful information. Data is captured from the UofS Course Catalogue.

• Technology used: Python, Django, React.js, MySQL, and HTML/CSS.
• Method: We scrape the data from the course catalogue using beautiful soup. Then, we clean the data and store it in our MySQL database. Finally, users search and interact with courses on the front-end, making API calls to the server to capture relevant data from the database.

Preview

Installation Steps

All done in the terminal

Client

• Note: MAKE SURE TO PERFORM THIS STEPS IN THE CLIENT FOLDER

• Node.js - Install node.js version 19.1.0 from https://nodejs.org/en/

• React.js

  • Install react using npx create-react-app
  • Install all the package.json dependencies using npm install
    • npm install @material-ui/core
    • npm install @mui/icons-material
    • npx json-server -p 3000 -w data/db.json
    • npm install axios npm install --save react-helmet
    • npm install @material-ui/core
    • npm install @mui/icons-material
    • npx json-server -p 3000 -w data/db.json
    • npm install axios
    • npm install --save react-helmet

Server

• Note: MAKE SURE TO PERFORM THIS STEPS IN THE SERVER FOLDER

• Django Dependencies

  • Install Django

    • pip install django

    • pip3 install djangorestframework-jsonapi

    • pip install mysqlclient

    • pip3 install djangorestframework

• Non-Django (Webscraping) Dependencies

  • Note: MAKE SURE TO DO THIS IN THE DIRECTORY relatable/server/webscraping
  • pip install bs4
  • pip install requests
  • pip install regex
  • pip install nltk

Database

• Installing MySQL

  • Install the community version from https://www.mysql.com/downloads/ (Go to Community addition at the bottom)
  • Download the “MySQL Installer for Windows”
    • Download the second one
  • In the MySQL installer
    • Select the developer version
    • Ignore the warnings
    • Install everything by clicking execute
    • Click next through everything
    • Create a root password; root is the main user
  • Click next through everything and execute
  • In the “Connect To Server” page, type in your password
  • Install the MySQL Workbench for a GUI. You can use the terminal to interact with MySQL if sought; in this case, add MySQL to your device's PATH variable (tutorial)

• Setting up the database

  • Create a new user: CREATE USER ‘username’@‘localhost’
  • Create a new database (keep track of the name): CREATE DATABASE 'name'

• Connecting django to the server

  • Add database details in the corresponding server .env file (see below)

Setting up .env files

• There are three .env files to populate with corresponding data
  • Create a .env file in Relatable/server/webscraping directory. Set the variable SERVER_URL={the url of the server running}
  • Create a .env file in the directory Relatable/server/server. Set the corresponding variables:
    • SECRET_KEY={the secret key of the database}
    • DATABASE_NAME={the name of the database}
    • DATABASE_USER={the user of the database}
    • DATABASE_PASSWORD={the password of the database}
    • DATABASE_HOST=localhost
    • DATABASE_PORT={whatever port you choose to use}
• Create a .env file in Relatable/client. Set the variable REACT_APP_SERVER_ENDPOINT={the url of the server running; must coincide with the one used in the webscraping folder}
• Restart your IDE

Running the application

• Running the server
  • CD to the server directory, then use the command python manage.py runserver
  • Note: The first you make changes to models.py or settings.py, you would have to do python manage.py makemigrations and then python manage.py migrate
• Scraping data and updating the database
  • CD to the server directory
  • Run (i.e., python path_to_file) the following files inside the webscraping directory (note, the first two files will take 10-20 minutes each as they pull 4609 courses from UofS's webpages):
    • initializeCourse.py
    • initializeDependency.py
    • initializePrerequisite.py
• Running the client
  • CD to the client directory
  • Use the command npm start

RESTART YOUR IDE BEFORE RUNNING OR IF YOU'VE DONE ALL THE STEPS CURRENTLY AND IT DOESN'T STILL WORK