Python · 机器学习 员工流失预测

提示：前言
Python · 机器学习 员工流失预测

---

提示：写完文章后，目录可以自动生成，如何生成可参考右边的帮助文档

---

前言

关于数据集
揭示导致员工流失的因素并探讨重要问题，例如“按工作角色和流失情况显示离家距离的细目分类”或“按教育程度和流失情况比较平均月收入”。这是一个由 IBM 数据科学家创建的虚构数据集。

学历
1‘大专以下’
2‘大专’
3‘本科’
4‘硕士’
5‘博士’

环境满意度
1“低”
2“中”
3“高”
4“非常高”

工作参与度
1 ‘低’
2 ‘中等’
3 ‘高’
4 ‘非常高’

工作满意度
1 ‘低’
2 ‘中等’
3 ‘高’
4 ‘非常高’

PerformanceRating
1 ‘低’
2 ‘好’
3 ‘优秀’
4 ‘优秀’

关系满意度
1 ‘低’
2 ‘中等’
3 ‘高’
4 ‘非常高’

WorkLifeBalance
1 ‘差’
2 ‘好’
3 ‘更好’
4 ‘最好’

数据集：https://www.kaggle.com/datasets/pavansubhasht/ibm-hr-analytics-attrition-dataset

---

提示：以下是本篇文章正文内容，下面案例可供参考

一、导入包

import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

import sklearn
from sklearn.model_selection import train_test_split, RepeatedKFold
from sklearn.metrics import roc_auc_score
from sklearn.preprocessing import StandardScaler

%matplotlib inline

TRAIN_CSV = '/kaggle/input/playground-series-s3e3/train.csv'
TEST_CSV = '/kaggle/input/playground-series-s3e3/test.csv'
EXTERNAL_CSV = '/kaggle/input/ibm-hr-analytics-attrition-dataset/WA_Fn-UseC_-HR-Employee-Attrition.csv'

二、数据准备阶段

2.1. 读取数据

首先，我们读取 CSV 文件并将所有数据放入单个数据框中。 虽然我们无法访问测试数据的 Attrition 特征，但通过这种数据集组合，我们可以使用其他特征之间的关系。

测试列表示数据集

target = 'Class'

df1 = pd.read_csv(TRAIN_CSV)
df1['test'] = 0
df1['gen'] = 1

df2 = pd.read_csv(TEST_CSV)
df2['test'] = 1
df2['gen'] = 1

df3 = pd.read_csv(EXTERNAL_CSV)
df3['test'] = 0
df3['gen'] = 0

#df = pd.concat([df1, df2]) # Do not include the original dataset
#df = pd.concat([df1, df2, df3[df3[target] == 1]])  # Only incorporate items with Class==1
df = pd.concat([df1, df2, df3])

df.id.fillna(-1, inplace=True)
df.id = df.id.astype(int)

df.reset_index(inplace=True)

df

三、检查缺失值

缺失值处理

# Histogram 
pd.isna(df).sum()

幸运的是，我们在 EmployeeNumber 列中只有缺失值。 我们还注意到，此列仅填充在原始数据集中，而我们在生成的数据集中没有它，因此必须将其删除。

df[pd.notna(df['EmployeeNumber'])].test.unique()

让我们分别检查每一列

Attrition 这是我们的目标专栏
我们将“是”/“否”值映射到 1/0。

df.Attrition = df.Attrition.map({'No' : 0, 'Yes' : 1, 0 : 0, 1 : 1})
df.Attrition.value_counts()

四、数值特征

如果列有许多唯一值，我们将其视为数字列。 有些列的唯一值很少，必须单独考虑

num_columns = ['Age', 'DailyRate', 'DistanceFromHome', 'HourlyRate', 
               'MonthlyIncome', 'MonthlyRate', 'TotalWorkingYears', 
               'YearsAtCompany', 'YearsInCurrentRole', 'YearsSinceLastPromotion',
               'YearsWithCurrManager']

fig, axs = plt.subplots(ncols=3, nrows=4, figsize=(24,18))
for n, col in enumerate(num_columns):
    ax = axs[n // 3, n % 3]
    sns.histplot(data=df[col], ax=ax);

五、分类特征

如果列具有字符串值或数值，但其中很少有不同，我们将其视为分类列。 我们打印每列的值、目标的平均值和具有每个值的项目数。

def explore_categorical_value(df, col, target='Attrition'):
    print(f'{col}')
    values = df[col].unique()
    values.sort()
    for value in values:
        select = df.loc[df[col] == value]
        mn = select[target].mean()
        cnt = len(select)
        cnt_test = len(select[select.test == 1])
        
        print(f'\t{value:16}\t{mn:.3f}\t{cnt}\t{cnt_test}')
        
        
cat_columns = ['BusinessTravel', 'Department', 'Education', 'EducationField', 
               'EmployeeCount', 'EnvironmentSatisfaction', 'Gender', 'JobInvolvement',
               'JobLevel', 'JobRole', 'JobSatisfaction', 'MaritalStatus', 
               'NumCompaniesWorked', 'Over18', 'OverTime', 'PercentSalaryHike', 
               'PerformanceRating', 'RelationshipSatisfaction', 'StandardHours', 
               'StockOptionLevel', 'TrainingTimesLastYear', 'WorkLifeBalance', 'test']

print(f'\t                  \tMean \tCount\tIn test')
for col in cat_columns:
    explore_categorical_value(df, col)

让我们看一下具有数值的列：
Attrition 的均值对 Education、EnvironmentSatisfaction、JobInvolvement 和 RelationshipSatisfaction 存在单调依赖性。 这些列很可能被视为数字。
对于JobLevel, JobSatisfaction, NumCompaniesWorked, PercentSalaryHike, StockOptionLevel, TrainingTimesLastYear, WorkLifeBalance 这样的依赖性不明显，我们要测试一下。PerformanceRating 只有 2 个值，所以我们认为它是分类的

此外，Employee Count、Over 18 和 Standard Hours 是无用的列，因为所有值都相同； 我们必须放弃它

columns_to_drop = ['EmployeeCount', 'Over18', 'StandardHours']

columns_numeric = ['Age', 'DailyRate', 'DistanceFromHome', 'HourlyRate', 'MonthlyIncome', 
                   'MonthlyRate', 'TotalWorkingYears', 'YearsAtCompany', 'YearsInCurrentRole', 
                   'YearsSinceLastPromotion', 'YearsWithCurrManager',
                   'Education', 'EnvironmentSatisfaction', 'JobInvolvement', 'RelationshipSatisfaction']

columns_categorical = ['BusinessTravel', 'Department', 'EducationField', 
                       'Gender', 'JobRole', 'MaritalStatus', 'OverTime', 
                       'PerformanceRating', 'gen']

columns_to_test_type = ['JobLevel', 'JobSatisfaction', 'NumCompaniesWorked', 'PercentSalaryHike', 
                        'StockOptionLevel', 'TrainingTimesLastYear', 'WorkLifeBalance']

“我们必须测试”列中的异常值：
在训练数据集中：

1. JobLevel 列有一个值为 7 的离群值

在测试数据集中：

1. NumCompaniesWorked 列有一个值为 17 的离群值
2. StockOptionLevel 列有一个值为 4 的离群值

如果我们将这些列视为数字，我们现在不必对异常值做任何事情。 但是对于分类列来说有一个问题。 让我们在下一个版本的笔记本中考虑它；

六、转换数据

df.drop(columns_to_drop, axis=1, inplace=True)

这里有一些空间来试验将列设置为分类或数字

def as_cat(col):
    if not col in columns_categorical:
        columns_categorical.append(col)
    if col in columns_numeric:
        columns_numeric.remove(col)

def as_num(col):
    if not col in columns_numeric:
        columns_numeric.append(col)
    if col in columns_categorical:
        columns_categorical.remove(col)

as_cat('JobLevel')
as_cat('JobSatisfaction')
as_cat('NumCompaniesWorked')
as_cat('PercentSalaryHike') #!
as_cat('StockOptionLevel') #!
as_cat('TrainingTimesLastYear')
as_num('WorkLifeBalance')

当我们将 NumCompaniesWorked 和 StockOptionLevel 设置为分类时，我们需要处理测试数据中的异常值。 一种简单的方法是用最接近的值替换。

df.loc[df.NumCompaniesWorked==17, 'NumCompaniesWorked'] = 9
df.loc[df.StockOptionLevel==4, 'StockOptionLevel'] = 3

for col in columns_categorical:
    values = df[col].unique()
    replace = {value : df.loc[df[col] == value, 'Attrition'].mean() for value in values}
    df[col] = df[col].map(replace)

七、模型预测

进行预测的简单模型
现在我们不执行复杂的参数优化。

7.1 定义一些有用的函数来快速测试模型

def get_numpy_arrays(data):
    X = data.drop(['id', 'test', 'Attrition'], axis=1).to_numpy()
    y = data.Attrition.to_numpy()
    
    scaler = StandardScaler()
    X = scaler.fit_transform(X)
    
    return X, y

def ensamble_pred(clfs, X):
    pred = 0
    for clf in clfs:
        pred += clf.predict_proba(X)[:, 1] if sklearn.base.is_classifier(clf) else clf.predict(X)
    return pred

def print_validation_score(clfs):
    pred = ensamble_pred(clfs, X_test)
    score = roc_auc_score(y_test, pred)
    print(f'\tValidation score = {score}')
    return score

def examine_clf(X_train_val, y_train_val, clf, rkf, verbose=None):
    avg_score = 0
    clfs = []
    for n, (train_index, val_index) in enumerate(rkf.split(X_train_val)):
        X_train = X_train_val[train_index]
        y_train = y_train_val[train_index]
        X_val = X_train_val[val_index]
        y_val = y_train_val[val_index]

        clf = sklearn.base.clone(clf)
        if verbose is None:
            clf.fit(X_train, y_train)
        else:
            clf.fit(X_train, y_train, verbose=verbose)
        pred = clf.predict_proba(X_val)[:, 1] if sklearn.base.is_classifier(clf) else clf.predict(X_val)
                
        score = roc_auc_score(y_val, pred)

        avg_score += score
        clfs.append(clf)

        #print(f"Fold {n}: {score:.3f}")

    avg_score /= (n+1)
    return clfs, avg_score

def examine_clf_print(X_train_val, y_train_val, clf, rkf, verbose=None):
    clfs, avg_score = examine_clf(X_train_val, y_train_val, clf, rkf, verbose=verbose)
    
    print(clf)
    print(f'\tCV avg score = {avg_score}')
    print_validation_score(clfs)
    
    return clfs

7.2 划分数据集

data = df[df.test != 1].copy()
data_submit = df[df.test == 1].copy()

X, y = get_numpy_arrays(data)
X_submit, y_submit = get_numpy_arrays(data_submit)

# Leave some data for testing
X_train_val, X_test, y_train_val, y_test = train_test_split(X, y, test_size=0.2, random_state=1)

all_clfs = []

7.3 测试模型

7.3.1. 线性模型：Ridge 和 Lasso 回归

from sklearn.linear_model import Lasso, Ridge
    
rkf = RepeatedKFold(n_splits=5, n_repeats=1, random_state=0)
lasso_clfs = examine_clf_print(X_train_val, y_train_val, Lasso(alpha=1e-5), rkf)
ridge_clfs = examine_clf_print(X_train_val, y_train_val, Ridge(alpha=1), rkf)

7.3.2. 随机森林回归器和分类器

from sklearn.ensemble import RandomForestRegressor, RandomForestClassifier

rfr_clfs = examine_clf_print(X_train_val, y_train_val, RandomForestRegressor(n_estimators=300, max_depth=10), rkf)
rfc_clfs = examine_clf_print(X_train_val, y_train_val, RandomForestClassifier(n_estimators=400, max_depth=10), rkf)

7.3.3. GB 回归器和分类器

from sklearn.ensemble import GradientBoostingRegressor, GradientBoostingClassifier

gbr_clfs = examine_clf_print(X_train_val, y_train_val, GradientBoostingRegressor(n_estimators=180, max_depth=2), rkf)
gbc_clfs = examine_clf_print(X_train_val, y_train_val, GradientBoostingClassifier(n_estimators=180, max_depth=2), rkf)

7.3.4. XGB

from xgboost import XGBRegressor, XGBClassifier

xgbr_clfs = examine_clf_print(X_train_val, y_train_val, XGBRegressor(n_estimators=100, max_depth=2), rkf)
xgbc_clfs = examine_clf_print(X_train_val, y_train_val, XGBClassifier(n_estimators=100, max_depth=2), rkf)

7.3.5. CatBoost regressor and classifier

from catboost import CatBoostRegressor, CatBoostClassifier
cbr = CatBoostRegressor(iterations=200, 
                    depth=2, 
                    learning_rate=0.1, 
                    loss_function='RMSE')

cbc = CatBoostClassifier(iterations=200, 
                    depth=2, 
                    learning_rate=0.1, 
                    loss_function='Logloss')

cbr_clfs = examine_clf_print(X_train_val, y_train_val, cbr, rkf, verbose=False)
cbc_clfs = examine_clf_print(X_train_val, y_train_val, cbc, rkf, verbose=False)

7.3.6. LGB

from lightgbm import LGBMClassifier, LGBMRegressor
lgbr = LGBMRegressor(n_estimators = 1000,
                        max_depth = 2,
                        learning_rate = 0.01,
                        num_leaves = 10,
                        lambda_l1 = 3,
                        lambda_l2 = 3,
                        bagging_fraction = 0.8,
                        feature_fraction = 0.8,
                        verbose=0)

lgbc = LGBMClassifier(n_estimators = 1000,
                        max_depth = 2,
                        learning_rate = 0.01,
                        num_leaves = 10,
                        lambda_l1 = 3,
                        lambda_l2 = 3,
                        bagging_fraction = 0.8,
                        feature_fraction = 0.8,
                        verbose=0)

lgbr_clfs = examine_clf_print(X_train_val, y_train_val, lgbr, rkf, verbose=False)
lgbc_clfs = examine_clf_print(X_train_val, y_train_val, lgbc, rkf, verbose=False)

八、 混合所有模型并检查结果

all_clfs = lasso_clfs + ridge_clfs + 3*rfc_clfs + gbc_clfs + xgbc_clfs + 5*cbc_clfs + lgbc_clfs

print_validation_score(all_clfs)

不是很好，但也不是很糟糕！

九、 预测并提交

data_submit['Attrition'] = ensamble_pred(all_clfs, X_submit)
data_submit.drop(data_submit.columns.difference(['id','Attrition']), axis=1, inplace=True)
data_submit.to_csv('submission.csv', index=False)

!head submission.csv

参考链接

https://www.kaggle.com/code/kdmitrie/pgs33-eda-basic-test-of-models-blending