mikolajmydel / projekt-ibm Goto Github PK

Use the best model (top performer) trained in you notebook code (#3) and measure adversarial training and test score.

Think how to illustrate simulated attack strength, how adversarial samples differ from original data.

Hint:

• Use pipeline generated by AutoAI to prepare data set in form of numerical matrix (skip the last step in the pipeline and use transform method).

ART Demo:
https://art-demo.mybluemix.net/

ART Git repo:
https://github.com/Trusted-AI/adversarial-robustness-toolbox

Example for random forest classifier:
https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/notebooks/classifier_scikitlearn_RandomForestClassifier.ipynb

Using data set describing Marvel characters developed under #1, run AutoAI experiment and compare trained models with the model created on you own. Finally, deploy the model in Machine Learning service and use scoring endpoint to make predictions.

• Use Data Refinery to profile and visualize your data set describing Marvel characters developed under #1. If needed refine data and run job to generate final set.
• Create and run AutoAI experiment. From the best performers select 3 models and store them as notebooks in your project.
• Run stored notebooks and compare evaluation metrics for trained models. Compare them with metrics produced in AutoAI experiment. Why are they slightly different?
• Try to modify the training pipeline or propose you own to get better evaluation results.
• Select the best model (top performer) and store it in the deployment space. Deploy the model as a web service (scoring endpoint) using code in a nodebook.
• Optional: use (jupyter widgets)[https://ipywidgets.readthedocs.io/en/latest/#] in JupyterLab running on your local machine to prepare interactive form with controls allowing to change Marvel character features and visualize prediction results. If you will you can use any other tools to prepare interactive UI application leveraging the scoring endpoint. Rest API docs: https://cloud.ibm.com/apidocs/machine-learning#deployments-compute-predictions

Good luck!

Your first task is to prepare a data set for machine learning (ML) model, capable of bad Marvel characters detection.

• Use publicly available data: https://www.kaggle.com/datasets/dannielr/marvel-superheroes. You may need to create Kaggle account to download data.
• Analise available data sets and choose those, which may be useful for bad characters detection.
• Join selected data sets and prepare a single one that will feed your training process.
• Identify labels that would be used for characters classification.
• Analyze features and classes distribution.
• Remove redundant features and clean data set, e.g. get rid of missing values, where needed.

Acceptance Criteria:

• Python code in form of a notebook showing all the steps listed above and pushed to this repository.

Materials:

• https://www.ibm.com/cloud/learn/supervised-learning
• https://pandas.pydata.org/
• https://www.kaggle.com/code/kanncaa1/feature-selection-and-data-visualization/notebook
• https://towardsdatascience.com/data-visualization-for-machine-learning-and-data-science-a45178970be7
• Marvel Comics