231 lines
6.1 KiB
Plaintext
231 lines
6.1 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f8a8990c-d09a-4fac-8e67-6a37ebaf056f",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Programming Exercise: Comparison of Regulations\n",
|
|
"Your task is to train an MLP for the classification of the iris data set using different regularization methods.\n",
|
|
"You can use the given libraries, but you can also use other libraries.\n",
|
|
"Set all seeds to 42."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ac54e46e-ce27-4d84-a29b-21442baee5f1",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Please enter your names\n",
|
|
"name = \"\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "62ec6feb-72cc-4431-9822-005a2200b217",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import random\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"import numpy as np\n",
|
|
"import tensorflow as tf\n",
|
|
"from sklearn.datasets import load_iris\n",
|
|
"from sklearn.model_selection import train_test_split\n",
|
|
"from sklearn.preprocessing import StandardScaler\n",
|
|
"\n",
|
|
"SEED = 42\n",
|
|
"os.environ[\"PYTHONHASHSEED\"] = str(SEED)\n",
|
|
"random.seed(SEED)\n",
|
|
"np.random.seed(SEED)\n",
|
|
"tf.random.set_seed(SEED)\n",
|
|
"\n",
|
|
"#load data\n",
|
|
"iris = load_iris()\n",
|
|
"X = iris.data \n",
|
|
"y = iris.target \n",
|
|
"\n",
|
|
"X = StandardScaler().fit_transform(X)\n",
|
|
"\n",
|
|
"X_train, X_val, y_train, y_val = train_test_split(\n",
|
|
" X, y, test_size=0.3, stratify=y, random_state=SEED\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ea6afe4b-18ce-4778-a786-1494ece331c3",
|
|
"metadata": {},
|
|
"source": [
|
|
"# MLP (2 pts)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "546ce278-188e-40eb-b5d8-d625b372cf3a",
|
|
"metadata": {},
|
|
"source": [
|
|
"First, implement an MLP with an input of 4, a hidden size of 16, and an output of 3 (a two-layer MLP).\n",
|
|
"For the first layer, use the ReLU function, and for the last layer, the softmax function."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "96acf91f-b9f2-413a-9f23-7a677d6ef0d3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9db21233-a43b-4a3f-8d26-847c5b9381ad",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Train MLP without any regularization (1 pt)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bbd429f2-fee9-47d7-aee8-d6263d42b845",
|
|
"metadata": {},
|
|
"source": [
|
|
"Train an MLP with the given train/validation split for 200 epochs with a batch size of 16.\n",
|
|
"Track the train loss, train accuracy, validation loss, and validation accuracy for each epoch. (e.g., in four arrays) "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "3038c352-b657-4d2c-a916-174c25582858",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "95be0757-fb8a-4532-9272-e2a0cd2ca704",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Train an MLP with dropout (1 pts)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fb615a7b-06d0-416e-92df-a359408796cc",
|
|
"metadata": {},
|
|
"source": [
|
|
"Train another MLP on the given train/validation split for 200 epochs with a batch size of 16 and dropout of 0.6.\n",
|
|
"Track the train loss, train accuracy, validation loss, and validation accuracy for each epoch. (e.g., in four arrays)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "82e0d61e-f58d-46a5-addd-587ab92bd5b3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b4f5d2b1-d9a8-4b2c-9f04-2c9d6fe744f4",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Train an MLP with the L2 Regularization (1 pts)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4dd894be-0a6e-4fa3-b905-a36ecb67aabb",
|
|
"metadata": {},
|
|
"source": [
|
|
"Train another MLP on the given train/validation split for 200 epochs with a batch size of 16 and the L2 Regularization with gamma = 0.02.\n",
|
|
"Track the train loss, train accuracy, validation loss, and validation accuracy for each epoch. (e.g., in four arrays)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9ea07dcc-a49d-48ad-b343-ebbaee68e960",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "38c93494-8ba1-479e-93ac-8c53577737ac",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Train an MLP with early stopping and a patience of 20 (1 pts)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3accc819-5712-4e78-90ef-de89a33e7917",
|
|
"metadata": {},
|
|
"source": [
|
|
"Train another MLP on the given train/validation split for 200 epochs with a batch size of 16 and use early stopping with a patience of 20.\n",
|
|
"Track the train loss, train accuracy, validation loss, and validation accuracy for each epoch. (e.g., in four arrays) Here it can stop earlier."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "0eacc33c-8549-490d-9000-2cd1aef62a46",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "02cafc93-6498-43fa-954e-cc531cce7a6f",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Plot the training of all four MLPs (2 pts)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f380b6eb-2950-424c-991f-6259a837d48e",
|
|
"metadata": {},
|
|
"source": [
|
|
"Create a plot for each MLP that shows its train loss, train accuracy, validation loss, and validation accuracy for each epoch. The plot for the MLP with early stopping can stop earlier."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "23cea99d-c226-4429-8884-dd2cc20c76c8",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.12"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|