drkostas / bert-rinehart-novels

BERT-based question answering/reading comprehension methods on Rinehart Novels.

Rinehart Analysis Question Answering

About

Current Issues, Assignment

Rinehart Analysis using Word Vectors for NLP (ECE-617) Project 2.

Libraries Overview

All the libraries are located under <project root>/nlp_libs

• <project root>/nlp_libs/books: This project's code (imported later) -<project root>/nlp_libs/configuration: Class that creates config objects from yml files
• <project root>/nlp_libs/fancy_logger: Logger that can be used instead of prints for text formatting (color, bold, underline etc)

Where to put the code

• Place the preprocessing functions/classes in nlp_libs/books/preprocessing.py
• The custom word embeddings functions/classes (task 1) in nlp_libs/books/word_embeddings.py (separate class)
• The pretrained word embeddings functions/classes (task 2) in nlp_libs/books/word_embeddings.py (separate class)
• The functions/classes (if any) that compare the results (tasks 3, 4, 5) in nlp_libs/books/compare_statistics.py
• Any plotting related functions in nlp_libs/books/plotter.py

The code is reloaded automatically. Any class object needs to reinitialized though.

Table of Contents

• About
  • Libraries Overview
  • Where to put the code
• Prerequisites
• Bootstrap Project
• Running the code using Jupyter + Configuration + Local Jupyter + Google Collab
• Adding New Libraries
• TODO
• License

Prerequisites

You need to have a machine with Python >= 3.9 and any Bash based shell (e.g. zsh) installed. Having installed conda is also recommended.

$ python3.9 -V
Python 3.9.7

$ echo $SHELL
/usr/bin/zsh

Bootstrap Project

All the installation steps are being handled by the Makefile.

If you want to use conda run:

$ make install

If you want to use venv run:

$ make install env=venv

Using Jupyter

Modifying the Configuration

You may need to configure the yml file. There is an already configured yml file under confs/proj_3.yml.

Local Jupyter

First, make sure you are in the correct virtual environment:

$ conda activate rinehartAnalysis_wordVectors

$ which python
/home/<your user>/anaconda3/envs/rinehartAnalysis_wordVectors/bin/python

To use jupyter, first run jupyter:

jupyter notebook

And open the main.ipynb.

Google Collab

Just Open this Google Collab Link.

Adding New Libraries

If you want to add a new library (e.g. a Class) in the project you need to follow these steps:

1. Create a new folder under "<project root>/nlp_libs" with a name like my_lib
2. Create a new python file inside it with a name like my_module.py
3. Paste your code inside it
4. Create a new file name init.py
5. Paste the following code inside it:

    """<Library name> sub-package."""
    
    from .<Module name> import <Class Name>
    
    __email__ = "[email protected], kgeorgio.vols.utk.edu, [email protected]"
    __author__ = "jeanmerlet, drkostas, LaneMatthewJ"
    __version__ = "0.1.0"

6. Open <project root>/nlp_libs/__init__.py
7. Add the following line: from nlp_libs.<Module name> import <Class Name>
8. (Optional) Rerun make install or python setup.py install

TODO

Read the TODO to see the current task list.

License

This project is licensed under the MIT License - see the LICENSE file for details.