机器学习之KNN&心脏病预测案例
·
1、KNN
1.1、KNN工作原理
K-Nearest-Neighbors，是监督学习算法，核心思想是通过计算给定样本与数据集中所有样本的距
离，找到距离最近的K个样本，然后根据这K个样本的类别或值来预测当前样本的类别或值。
先计算距离，再选择最近距离的K个样本
K值：K值的选择对结果影响很大。K值过小容易过拟合，K值过大则可能欠拟合。
1.2、KNN_API分类器
K邻近类里可以设置权重weights = distance，这样就会根据距离分类，而不会出现如过K=2一人一
票之后的按原顺序分配
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=2)
X = [[2, 1], [3, 1], [1, 4], [2, 6]] # 特征
y = [0, 0, 1, 1] # 标签
knn.fit(X,y)
knn_pred = knn.predict([[4,9]])
print(knn_pred)
[1]1.3、KNN_API回归器
from sklearn.neighbors import KNeighborsClassifier, KNeighborsRegressor
knn = KNeighborsRegressor(n_neighbors=2,weights='distance')
X = [[2, 1], [3, 1], [1, 4], [2, 6]] # 特征
y = [0.5, 0.33, 4, 3]
knn.fit(X,y)
y_pred = knn.predict([[4,9]])
print(y_pred)
[3.38208553]2、常见测距方法（了解）
欧式距离（两点之间的直线距离）
曼哈顿距离（参考曼哈顿街道）
切比雪夫距离（两点各坐标数值差的最大值）
闵可夫斯基距离（是一种用于度量多维空间中两点间距离的通用方法）
3、特征转换——归一化、标准化
3.1、归一化
将数据按比例缩放到一个固定范围[𝑚𝑖𝑛,𝑚𝑎𝑥]
消除量纲差异，避免模型被大范围特征主导，加速模型收敛，适配特定模型需求
这里的例子是按列（按特征）归一化的
from sklearn.preprocessing import MinMaxScaler
X = [[2, 1], [3, 1], [1, 4], [2, 6]]
scaler = MinMaxScaler(feature_range=(-1,1)).fit_transform(X)
print(scaler)
[[ 0. -1. ]
[ 1. -1. ]
[-1. 0.2]
[ 0. 1. ]]3.2、标准化
将数据调整为均值为0、标准差为1的标准分布。
from sklearn.preprocessing import StandardScaler
X = [[2, 1], [3, 1], [1, 4], [2, 6]]
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(X_scaled)
[[ 0. -0.94280904]
[ 1.41421356 -0.94280904]
[-1.41421356 0.47140452]
[ 0. 1.41421356]]4、KNN案例——心脏病预测
4.1、代码
数据集https://www.kaggle.com/datasets/johnsmith88/heart-disease-dataset
这里用到了ColumnTransformer列转换器，参数是transformers=[]，里面传入针对不同特征类型处
理特征的方式，独热编码里的drop = 'first'可以默认去掉第一列避免多重共线性，score =
knn.score(x_test,y_test)这是一个直接查看accuracy的方法
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler,OneHotEncoder
from sklearn.neighbors import KNeighborsClassifier
dataset = pd.read_csv('data/heart_disease.csv')
dataset.dropna(inplace=True)
x = dataset.drop('是否患有心脏病',axis=1)
y = dataset['是否患有心脏病']
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3,random_state=42)
numerical_features = ["年龄", "静息血压", "胆固醇", "最大心率", "运动后的ST下降", "主血管数量"]
categorical_features = ["胸痛类型", "静息心电图结果", "峰值ST段的斜率", "地中海贫血"]
binary_features = ["性别", "空腹血糖", "运动性心绞痛"]
column_transformer = ColumnTransformer(transformers=[
("num",StandardScaler(),numerical_features),
('cat',OneHotEncoder(drop = 'first'),categorical_features),
('bin','passthrough',binary_features)
])
x_train = column_transformer.fit_transform(x_train)
x_test = column_transformer.transform(x_test)
knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(x_train,y_train)
score = knn.score(x_test,y_test)
print(score)
0.92532467532467534.2、模型保存
这是一个保存模型的方式，用来保存已经训练好的模型
import joblib
joblib.dump(knn, 'knn_model.joblib')
knn_loaded = joblib.load('knn_model.joblib')
print(knn_loaded.score(x_test,y_test))
y_pred = knn_loaded.predict(x_test[[10]])
print(y_pred,y_test[10])4.3、网格搜索（Grid Search）
通过遍历预定义的超参数组合，找到使模型性能最优的参数配置。通过自动化调参避免手动试错，
提高效率。sklearn.model_selection.GridSearchCV
import pandas as pd
from sklearn.model_selection import train_test_split,GridSearchCV
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler,OneHotEncoder
from sklearn.neighbors import KNeighborsClassifier
dataset = pd.read_csv('data/heart_disease.csv')
dataset.dropna(inplace=True)
x = dataset.drop('是否患有心脏病',axis=1)
y = dataset['是否患有心脏病']
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3,random_state=42)
numerical_features = ["年龄", "静息血压", "胆固醇", "最大心率", "运动后的ST下降", "主血管数量"]
categorical_features = ["胸痛类型", "静息心电图结果", "峰值ST段的斜率", "地中海贫血"]
binary_features = ["性别", "空腹血糖", "运动性心绞痛"]
column_transformer = ColumnTransformer(transformers=[
("num",StandardScaler(),numerical_features),
('cat',OneHotEncoder(drop = 'first'),categorical_features),
('bin','passthrough',binary_features)
])
x_train = column_transformer.fit_transform(x_train)
x_test = column_transformer.transform(x_test)
knn = KNeighborsClassifier()
params_grid = {'n_neighbors':list(range(1,11))} # KNN的K值
gs_cv = GridSearchCV(estimator=knn, param_grid=params_grid,cv = 10) # K折交叉验证cv
gs_cv.fit(x_train,y_train)
print(pd.DataFrame(gs_cv.cv_results_).to_string())
print(gs_cv.best_params_)
print(gs_cv.best_score_)
print(gs_cv.best_estimator_)
mean_fit_time std_fit_time mean_score_time std_score_time param_n_neighbors params split0_test_score split1_test_score split2_test_score split3_test_score split4_test_score split5_test_score split6_test_score split7_test_score split8_test_score split9_test_score mean_test_score std_test_score rank_test_score
0 0.000772 0.000087 0.142634 0.420856 1 {'n_neighbors': 1} 0.986111 1.000000 0.972222 0.986111 1.000000 0.986111 0.972222 0.985915 0.957746 0.943662 0.979010 0.016953 1
1 0.000873 0.000264 0.002216 0.000183 2 {'n_neighbors': 2} 0.944444 0.930556 0.916667 0.930556 0.958333 0.944444 0.902778 0.943662 0.901408 0.845070 0.921792 0.031117 2
2 0.000770 0.000159 0.002498 0.000352 3 {'n_neighbors': 3} 0.930556 0.833333 0.902778 0.861111 0.875000 0.916667 0.888889 0.845070 0.845070 0.774648 0.867312 0.043574 7
3 0.000732 0.000076 0.002418 0.000396 4 {'n_neighbors': 4} 0.902778 0.805556 0.888889 0.875000 0.888889 0.875000 0.875000 0.859155 0.816901 0.774648 0.856182 0.040169 10
4 0.000802 0.000137 0.002450 0.000418 5 {'n_neighbors': 5} 0.930556 0.805556 0.847222 0.916667 0.888889 0.902778 0.888889 0.901408 0.802817 0.760563 0.864534 0.054298 8
5 0.000753 0.000093 0.002428 0.000304 6 {'n_neighbors': 6} 0.916667 0.833333 0.888889 0.888889 0.888889 0.916667 0.902778 0.915493 0.802817 0.760563 0.871498 0.051363 5
6 0.000743 0.000078 0.002412 0.000298 7 {'n_neighbors': 7} 0.958333 0.875000 0.902778 0.930556 0.902778 0.902778 0.916667 0.915493 0.873239 0.788732 0.896635 0.042995 3
7 0.000717 0.000042 0.002428 0.000313 8 {'n_neighbors': 8} 0.944444 0.819444 0.888889 0.916667 0.902778 0.888889 0.930556 0.915493 0.845070 0.816901 0.886913 0.042873 4
8 0.000727 0.000057 0.002531 0.000371 9 {'n_neighbors': 9} 0.916667 0.819444 0.875000 0.902778 0.875000 0.875000 0.902778 0.887324 0.830986 0.816901 0.870188 0.034057 6
9 0.000811 0.000259 0.002538 0.000359 10 {'n_neighbors': 10} 0.888889 0.791667 0.861111 0.902778 0.875000 0.861111 0.875000 0.887324 0.845070 0.802817 0.859077 0.034666 9
{'n_neighbors': 1}
0.979010172143975
KNeighborsClassifier(n_neighbors=1)也可以增加超参数来计算
params_grid = {'n_neighbors':list(range(1,11)),'weights':['uniform','distance'],'p':[1,2]} # KNN的K值
{'n_neighbors': 9, 'p': 2, 'weights': 'distance'}
0.9887910798122066
KNeighborsClassifier(n_neighbors=9, weights='distance')使用最优模型去预测测试集数据
knn = gs_cv.best_estimator_
print(knn.score(x_test,y_test))
1.0
DAMO开发者矩阵，由阿里巴巴达摩院和中国互联网协会联合发起，致力于探讨最前沿的技术趋势与应用成果，搭建高质量的交流与分享平台，推动技术创新与产业应用链接，围绕“人工智能与新型计算”构建开放共享的开发者生态。
更多推荐
4
0- 0
已为社区贡献3条内容
所有评论(0)