#!/usr/bin/env python
# coding: utf-8

# <br />
# <br />
# 
# # Exercises: Day 3
# 
# <br />
# 
# This notebook contains exercises for Day 3. If you are taking this workshop for UCLA credits, you will need to submit this notebook and the one for yesterday as your homework until the second Monday from today to the instructor's email. Please submit BOTH `.ipynb` file and the HTML (`.html`) file converted from this notebook. (`File > Export Notebook As... > HTML`)
# 
# **Requirement: Complete at least one of the following exercises.**

# In[1]:


import numpy as np
import matplotlib.pyplot as plt


# ## Exercise 3: Hyperparameter tuning for penalized regression

# Consider the data and ridge regression algorithm with 20-degree polynomials as in the class. Tune the hyperparameter $\alpha$ using $K$-fold cross validation based on the $R^2$ measure. The steps would be similar to what we did with classifiers:

# ### 1) Load the datafile `Regression_Exercise_dataset.dat`

# ### 2) Split out test data.

# ### 3) Define a set of candidates for $\alpha$. Choose hyperparameter $\alpha$ for ridge regression using 4-fold cross validation, with mean squared error. Use a `RidgeCV` object. What is the best $\alpha$ value chosen?

# Note: Carefully read the documentation and select arguments for the object. You want to put `cv=4`, `scoring="neg_mean_squared_error"`. 

# ### 4) Define a set of candidates for $\alpha$. Choose hyperparameter $\alpha$ for lasso regression using 4-fold cross validation. Use a `LassoCV` object. What is the best $\alpha$ value chosen?

# ### 5) Compare the final performance of the two with the test set, in terms of mean squared error. 
# _Hint:_ use `metrics.mean_squared_error`.

# ## Exercise 4: Principal Component Analysis + K-means clustering

# Let's try PCA on the iris dataset.

# In[8]:


from sklearn.datasets import load_iris
iris_data = load_iris()


# We have 150 samples with four features, 

# In[9]:


print( iris_data.data.shape )


# And we have three types of iris: setosa, versicolor, and virginica.

# In[6]:


iris_data.target


# In[7]:


iris_data.target_names


# ### 1) Compute the covariance matrix.

# ### 2) Fit PCA.

# ### 3) How much of the variance is explained by each PC?  Visualize it. What are the principal components? 

# ### 4) Perform the PCA transformation. What is the covariance matrix on the transformed space?

# ### 5) Plot the samples on the first two principal components. Use different colors/markers for different kind of iris.

# ### 6) Perform K-means clustering with $K=3$ on the PCA-transformed space, and visualize the result using first two principal components.