ahhhz975 / bigdataproject Goto Github PK

One potential way to do this is to deeply understand the features in the dataset and explore it further.

The other important point here is that we should clearly talk to Sofvie and determine what exactly their goal is. For example, they might say we need only a method that classifies the data into 4 or 5 groups. In this case, we should implement one of the clustering methods mentioned in our project summary. In another scenario, assume they might say they have some pre-defined classes (groups) like (low, med, and high completion rates) and then they need to classify the employees in these classes.

Anyway, this has to be done before starting any other tasks.

To perform the classification or clustering task for classifying the employees based on their training completion rates, we need to determine two parameters: 1- The number of classes (or categories) that we expect our method to break down the data into 2- The classes (categories) that we expect our method to break down the data into

If the team comes up with Clustering-based methods (unsupervised), then we need only the first parameter (the number of classes). Otherwise, if we come up with classification-methods (supervised), then we need both parameters to implement the method.

The dataset shared by Sofvie now is an online dataset that needs to be saved in our project directory to easily work with. To do that, we need a simple "dataLoader" function to automatically load the dataset from the Sofvie domain and then store it in the directory of the project.

Whether we decide on performing clustering (unsupervised) methods or classification (supervised) methods, we need to implement a deep neural framework to classify the employees based on their training completion rates.

Note: Personally, I am thinking of an auto-encoder (encoder-decoder) architecture as a potential architectural design for our deep neural framework. In this architecture, we first extract unique features of the data using the encoder part of the architecture. This part is the common part in both clustering and classification methods. Then, we convert those features to some classes or a number of classes using the decoder part. This design of the decoder will be changed based on the task we will decide to work on (which could be either clustering or classification).