Predictors of Mental Health Illness

 Machine Learning Model Training

Created: 17/07/2023
Last update: 22/07/2023

This kernel is modified from the “Python Jupyter Notebook” by Laksara Abeysinghe

This is one of my kernel regrading of ML techniques. I'm trying to get better experiences though every concepts of ML by practicing and Kaggle is a good place to do it. Thanks Kaggle team for the data source.

Dataset Information

This dataset is from a 2014 survey that measures attitudes towards mental health and frequency of mental health disorders in the tech workplace. You are also encouraged to analyze data from the Kaggle website. Found from here.

Content

This dataset contains the following data:
• Timestamp
• Age
• Gender
• Country
• state: If you live in the United States, which state or territory do you live in?
• self_employed: Are you self-employed?
• family_history: Do you have a family history of mental illness?
• treatment: Have you sought treatment for a mental health condition?
• work_interfere: If you have a mental health condition, do you feel that it interferes with your work?
• no_employees: How many employees does your company or organization have?
• remote_work: Do you work remotely (outside of an office) at least 50% of the time?
• tech_company: Is your employer primarily a tech company/organization?
• benefits: Does your employer provide mental health benefits?
• care_options: Do you know the options for mental health care your employer provides?
• wellness_program: Has your employer ever discussed mental health as part of an employee wellness program?
• seek_help: Does your employer provide resources to learn more about mental health issues and how to seek help?
• anonymity: Is your anonymity protected if you choose to take advantage of mental health or substance abuse treatment resources?
• leave: How easy is it for you to take medical leave for a mental health condition?
• mentalhealthconsequence: Do you think that discussing a mental health issue with your employer would have negative consequences?
• physhealthconsequence: Do you think that discussing a physical health issue with your employer would have negative consequences?
• coworkers: Would you be willing to discuss a mental health issue with your coworkers?
• supervisor: Would you be willing to discuss a mental health issue with your direct supervisor(s)?
• mentalhealthinterview: Would you bring up a mental health issue with a potential employer in an interview?
• physhealthinterview: Would you bring up a physical health issue with a potential employer in an interview?
• mentalvsphysical: Do you feel that your employer takes mental health as seriously as physical health?
• obs_consequence: Have you heard of or observed negative consequences for coworkers with mental health conditions in your workplace?
• comments: Any additional notes or comments

Inspiration

The question worth exploring:
Can you predict whether a patient should be treated of his/her mental illness or not according to the values obtained in the dataset?

The original dataset is from Open Sourcing Mental Illness and can be downloaded here.

In [1]:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import warnings

warnings.filterwarnings('ignore')

In [2]:

df = pd.read_csv('survey.csv')
df.head()

Out[2]:

	Timestamp	Age	Gender	Country	state	self_employed	family_history	treatment	work_interfere	no_employees	...	leave	mental_health_consequence	phys_health_consequence	coworkers	supervisor	mental_health_interview	phys_health_interview	mental_vs_physical	obs_consequence	comments
0	2014-08-27 11:29:31	37	Female	United States	IL	NaN	No	Yes	Often	6-25	...	Somewhat easy	No	No	Some of them	Yes	No	Maybe	Yes	No	NaN
1	2014-08-27 11:29:37	44	M	United States	IN	NaN	No	No	Rarely	More than 1000	...	Don't know	Maybe	No	No	No	No	No	Don't know	No	NaN
2	2014-08-27 11:29:44	32	Male	Canada	NaN	NaN	No	No	Rarely	6-25	...	Somewhat difficult	No	No	Yes	Yes	Yes	Yes	No	No	NaN
3	2014-08-27 11:29:46	31	Male	United Kingdom	NaN	NaN	Yes	Yes	Often	26-100	...	Somewhat difficult	Yes	Yes	Some of them	No	Maybe	Maybe	No	Yes	NaN
4	2014-08-27 11:30:22	31	Male	United States	TX	NaN	No	No	Never	100-500	...	Don't know	No	No	Some of them	Yes	Yes	Yes	Don't know	No	NaN

5 rows × 27 columns

In [3]:

df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1259 entries, 0 to 1258
Data columns (total 27 columns):
 #   Column                     Non-Null Count  Dtype 
---  ------                     --------------  ----- 
 0   Timestamp                  1259 non-null   object
 1   Age                        1259 non-null   int64 
 2   Gender                     1259 non-null   object
 3   Country                    1259 non-null   object
 4   state                      744 non-null    object
 5   self_employed              1241 non-null   object
 6   family_history             1259 non-null   object
 7   treatment                  1259 non-null   object
 8   work_interfere             995 non-null    object
 9   no_employees               1259 non-null   object
 10  remote_work                1259 non-null   object
 11  tech_company               1259 non-null   object
 12  benefits                   1259 non-null   object
 13  care_options               1259 non-null   object
 14  wellness_program           1259 non-null   object
 15  seek_help                  1259 non-null   object
 16  anonymity                  1259 non-null   object
 17  leave                      1259 non-null   object
 18  mental_health_consequence  1259 non-null   object
 19  phys_health_consequence    1259 non-null   object
 20  coworkers                  1259 non-null   object
 21  supervisor                 1259 non-null   object
 22  mental_health_interview    1259 non-null   object
 23  phys_health_interview      1259 non-null   object
 24  mental_vs_physical         1259 non-null   object
 25  obs_consequence            1259 non-null   object
 26  comments                   164 non-null    object
dtypes: int64(1), object(26)
memory usage: 265.7+ KB

By seeing above output which informations of contain features of dataset, it can be seen all variables' data types are object version instead of Age of the persons. Altogether 27 features are there in Mental Health dataset with differnt type of Comments.

In [4]:

df.isnull().sum().sort_values(ascending=False)

Out[4]:

comments                     1095
state                         515
work_interfere                264
self_employed                  18
seek_help                       0
obs_consequence                 0
mental_vs_physical              0
phys_health_interview           0
mental_health_interview         0
supervisor                      0
coworkers                       0
phys_health_consequence         0
mental_health_consequence       0
leave                           0
anonymity                       0
Timestamp                       0
wellness_program                0
Age                             0
benefits                        0
tech_company                    0
remote_work                     0
no_employees                    0
treatment                       0
family_history                  0
Country                         0
Gender                          0
care_options                    0
dtype: int64

Four variables have some missing data such State, Work Interfere, Self Employed and Comments of persons.

In [5]:

plt.figure(figsize=(15,3))
sns.heatmap(df.isnull(), cbar=False)

Out[5]:

<AxesSubplot:>

White lines indicate the missing data of dataset.

Cleaning the NaN and Other Feature Engineering

In [6]:

# Cleaning 'Age' & Categorized Age Groups 

ageseries = pd.Series(df['Age'])
ageseries[ageseries<18] = df['Age'].median()
ageseries[ageseries>100] = df['Age'].median()
df['Age'] = ageseries

df['Age_Group'] = pd.cut(df['Age'], [0,20,30,45,55,100], labels=('0-20', '21-30', '31-45', '46-55', '55-100'), include_lowest=True)
print(df['Age_Group'].unique())

['31-45', '21-30', '46-55', '0-20', '55-100']
Categories (5, object): ['0-20' < '21-30' < '31-45' < '46-55' < '55-100']

In [7]:

# Cleaning 'self_employed'

df['self_employed'] = df['self_employed'].fillna(df['self_employed'].mode()[0])
print(df['self_employed'].unique())

['No' 'Yes']

In [8]:

# Cleaning 'work_interfere'

defaultString = 'NaN'
df['work_interfere'] = df['work_interfere'].fillna(defaultString)
df['work_interfere'] = df['work_interfere'].replace([defaultString], 'Don\'t know')
print(df['work_interfere'].unique())

['Often' 'Rarely' 'Never' 'Sometimes' "Don't know"]

In [9]:

#clean 'Gender'

#lower case all columm's elements
gender = df['Gender'].str.lower()
#print(gender)

#Select unique elements
gender = df['Gender'].unique()

#Made gender groups
male_str = ["male", "m", "male-ish", "maile", "mal", "male (cis)", "make", "male ", "man","msle", "mail", "malr","cis man", "Cis Male", "cis male"]
trans_str = ["trans-female", "something kinda male?", "queer/she/they", "non-binary","nah", "all", "enby", "fluid", "genderqueer", "androgyne", "agender", "male leaning androgynous", "guy (-ish) ^_^", "trans woman", "neuter", "female (trans)", "queer", "ostensibly male, unsure what that really means"]           
female_str = ["cis female", "f", "female", "woman",  "femake", "female ","cis-female/femme", "female (cis)", "femail"]

for (row, col) in df.iterrows():

    if str.lower(col.Gender) in male_str:
        df['Gender'].replace(to_replace=col.Gender, value='male', inplace=True)

    if str.lower(col.Gender) in female_str:
        df['Gender'].replace(to_replace=col.Gender, value='female', inplace=True)

    if str.lower(col.Gender) in trans_str:
        df['Gender'].replace(to_replace=col.Gender, value='trans', inplace=True)

#Get rid of bullshit
stk_list = ['A little about you', 'p']
df = df[~df['Gender'].isin(stk_list)]

print(df['Gender'].unique())

['female' 'male' 'trans']

Feature Visualizations

In [10]:

df_drp_col = [0,1,3,4,26]
droppedCol = df.drop(df.columns[df_drp_col], axis=1)
droppedCol.head()

for columns in droppedCol.columns:
    print(f"{columns}.............................................................................................................")
    categories = droppedCol[columns].value_counts()
    for category, count in categories.items():
        print(f"{category}   --   {count}")

Gender.............................................................................................................
male   --   991
female   --   247
trans   --   19
self_employed.............................................................................................................
No   --   1113
Yes   --   144
family_history.............................................................................................................
No   --   767
Yes   --   490
treatment.............................................................................................................
Yes   --   635
No   --   622
work_interfere.............................................................................................................
Sometimes   --   465
Don't know   --   264
Never   --   213
Rarely   --   173
Often   --   142
no_employees.............................................................................................................
6-25   --   290
26-100   --   289
More than 1000   --   282
100-500   --   176
1-5   --   160
500-1000   --   60
remote_work.............................................................................................................
No   --   883
Yes   --   374
tech_company.............................................................................................................
Yes   --   1029
No   --   228
benefits.............................................................................................................
Yes   --   475
Don't know   --   408
No   --   374
care_options.............................................................................................................
No   --   501
Yes   --   442
Not sure   --   314
wellness_program.............................................................................................................
No   --   842
Yes   --   227
Don't know   --   188
seek_help.............................................................................................................
No   --   646
Don't know   --   363
Yes   --   248
anonymity.............................................................................................................
Don't know   --   819
Yes   --   373
No   --   65
leave.............................................................................................................
Don't know   --   563
Somewhat easy   --   266
Very easy   --   204
Somewhat difficult   --   126
Very difficult   --   98
mental_health_consequence.............................................................................................................
No   --   490
Maybe   --   477
Yes   --   290
phys_health_consequence.............................................................................................................
No   --   925
Maybe   --   273
Yes   --   59
coworkers.............................................................................................................
Some of them   --   774
No   --   260
Yes   --   223
supervisor.............................................................................................................
Yes   --   514
No   --   393
Some of them   --   350
mental_health_interview.............................................................................................................
No   --   1008
Maybe   --   207
Yes   --   42
phys_health_interview.............................................................................................................
Maybe   --   557
No   --   500
Yes   --   200
mental_vs_physical.............................................................................................................
Don't know   --   576
Yes   --   341
No   --   340
obs_consequence.............................................................................................................
No   --   1075
Yes   --   182
Age_Group.............................................................................................................
31-45   --   622
21-30   --   557
46-55   --   42
0-20   --   22
55-100   --   14

Above output demonstrates all the categories of feature contains. By seeing that a brief idea can be taken. So, follow graphs describes much understanding visualization of categories.

In [11]:

# Plotting all selected variables

fig, axes = plt.subplots(nrows=12, ncols=2, figsize=(12, 26))

for i, column in enumerate(droppedCol.columns):
    ax = axes[i // 2, i % 2]  # Get the corresponding subplot axes
    categories = droppedCol[column].value_counts()

    # Plotting the category counts with 'cool' color palette
    sns.barplot(x=categories.index, y=categories.values, ax=ax, palette='cool')
    ax.set_title(column)
    ax.set_xlabel("Categories")
    ax.set_ylabel("Count")
    ax.tick_params(axis='x')

plt.tight_layout()
plt.show()

Age Distribution Visualization

In [12]:

plt.figure(figsize=(18,8))

# Raw Age 
plt.subplot(1,2,1)
plt.hist(df['Age'], bins=30, histtype = 'step')
plt.title('Age Distribution')
plt.xlabel('Age')

# Age Groups 
plt.subplot(1,2,2)
sns.barplot(x = df['Age_Group'].value_counts().index, y = df['Age_Group'].value_counts().values,
           palette = 'cool')
plt.title('Age Distribution')
plt.xlabel('Age Groups')
plt.show()

Most of persons are under 31-45 age group. Very less are under above 55 age.

In [13]:

# Gender Diversity

plt.figure(figsize=(15,10))

plt.subplot(2,1,1)
sns.barplot(x = df['Gender'].value_counts().index,
       y = df['Gender'].value_counts().values, palette='cool')
plt.title('Gender')


# By Age Groups

plt.subplot(2,1,2)
genderByAge = df.groupby(['Age_Group', 'Gender']).size().reset_index().rename(
    columns={0:'Num.of People'}).sort_values('Num.of People', ascending = False)
sns.barplot(x = 'Gender', y = 'Num.of People', data = genderByAge, hue='Age_Group', dodge = True,  palette='cool')
plt.title("Gender Diversity by Age Groups")
plt.show()

Model Building

Dropping useless features from the dataset.

In [14]:

df = df.drop(["Timestamp", "state", "Age", "Country"],axis=1)

In [15]:

df.head()

Out[15]:

	Gender	self_employed	family_history	treatment	work_interfere	no_employees	remote_work	tech_company	benefits	care_options	...	mental_health_consequence	phys_health_consequence	coworkers	supervisor	mental_health_interview	phys_health_interview	mental_vs_physical	obs_consequence	comments	Age_Group
0	female	No	No	Yes	Often	6-25	No	Yes	Yes	Not sure	...	No	No	Some of them	Yes	No	Maybe	Yes	No	NaN	31-45
1	male	No	No	No	Rarely	More than 1000	No	No	Don't know	No	...	Maybe	No	No	No	No	No	Don't know	No	NaN	31-45
2	male	No	No	No	Rarely	6-25	No	Yes	No	No	...	No	No	Yes	Yes	Yes	Yes	No	No	NaN	31-45
3	male	No	Yes	Yes	Often	26-100	No	Yes	No	Yes	...	Yes	Yes	Some of them	No	Maybe	Maybe	No	Yes	NaN	31-45
4	male	No	No	No	Never	100-500	Yes	Yes	Yes	No	...	No	No	Some of them	Yes	Yes	Yes	Don't know	No	NaN	31-45

5 rows × 24 columns

Lable Encoding

In [16]:

from sklearn.preprocessing import LabelEncoder
LE = LabelEncoder()

for i in df.columns:
    df[i] = LE.fit_transform(df[i])

Split Dataset into Train and Test

In [17]:

x = df.drop(["treatment"], axis=1)
y = df["treatment"]
x.shape, y.shape

Out[17]:

((1257, 23), (1257,))

In [18]:

from sklearn.model_selection import train_test_split
x_train,x_test,y_train,y_test = train_test_split(x,y, test_size=0.25, random_state=42)

1. Logistic Regression Model Training

In [19]:

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import accuracy_score

In [20]:

LG_model = LogisticRegression()

In [21]:

param1 = {'C':[0.1,2,5,10,15,20], 'penalty':['l1','l2'], 'max_iter':[100,200,300]}

In [22]:

model_1=GridSearchCV(LG_model, param_grid=param1, scoring='f1', cv=5)

In [23]:

model_1.fit(x_train,y_train)

Out[23]:

GridSearchCV(cv=5, estimator=LogisticRegression(),
             param_grid={'C': [0.1, 2, 5, 10, 15, 20],
                         'max_iter': [100, 200, 300], 'penalty': ['l1', 'l2']},
             scoring='f1')

In [24]:

print(model_1.best_params_)
best_LG_model = model_1.best_estimator_

{'C': 2, 'max_iter': 300, 'penalty': 'l2'}

In [25]:

LGy_pred = best_LG_model.predict(x_test)

In [26]:

accuracy_value = []
accuracy_value.append(accuracy_score(y_test,LGy_pred)*100)

print("Accuracy: ",accuracy_score(y_test,LGy_pred)*100)

Accuracy:  79.04761904761905

2. Decision Tree Classifier Model Training

In [27]:

from sklearn.tree import DecisionTreeClassifier

In [28]:

DT_model = DecisionTreeClassifier()

In [29]:

param2 = {
    'criterion': ['gini', 'entropy'],
    'max_depth': [None,5,10,15],
    'min_samples_split': [2,5,10],
    'min_samples_leaf': [1,2,3]
}

In [30]:

model_2=GridSearchCV(DT_model, param_grid=param2, cv=5)
model_2.fit(x_train,y_train)

Out[30]:

GridSearchCV(cv=5, estimator=DecisionTreeClassifier(),
             param_grid={'criterion': ['gini', 'entropy'],
                         'max_depth': [None, 5, 10, 15],
                         'min_samples_leaf': [1, 2, 3],
                         'min_samples_split': [2, 5, 10]})

In [31]:

print(model_2.best_params_)
best_DT_model = model_2.best_estimator_

{'criterion': 'entropy', 'max_depth': 5, 'min_samples_leaf': 1, 'min_samples_split': 2}

In [32]:

DTy_pred = best_DT_model.predict(x_test)

In [33]:

accuracy_value.append(accuracy_score(y_test,DTy_pred)*100)

print("Accuracy: ",accuracy_score(y_test, DTy_pred)*100)

Accuracy:  80.63492063492063

3. Random Forest Classifier Model Training

In [34]:

from sklearn.ensemble import RandomForestClassifier
RF_model = RandomForestClassifier()

In [35]:

param3 = {
    'n_estimators': [100,200],
    'max_depth': [None,5,10,15,20],
    'min_samples_split': [2,5,10],
    'min_samples_leaf': [1,2,3]
}

In [36]:

model_3=GridSearchCV(RF_model, param_grid=param3, cv=5)
model_3.fit(x_train,y_train)

Out[36]:

GridSearchCV(cv=5, estimator=RandomForestClassifier(),
             param_grid={'max_depth': [None, 5, 10, 15, 20],
                         'min_samples_leaf': [1, 2, 3],
                         'min_samples_split': [2, 5, 10],
                         'n_estimators': [100, 200]})

In [37]:

print(model_3.best_params_)
best_RF_model = model_3.best_estimator_

{'max_depth': 5, 'min_samples_leaf': 2, 'min_samples_split': 5, 'n_estimators': 100}

In [38]:

RFy_pred = best_RF_model.predict(x_test)

In [39]:

accuracy_value.append(accuracy_score(y_test,RFy_pred)*100)

print("Accuracy: ",accuracy_score(y_test, RFy_pred)*100)

Accuracy:  80.63492063492063

4. Support Vector Classifier Model Training

In [40]:

from sklearn.svm import SVC
svc_model = SVC()

In [41]:

param4 = {
    'C': [0.1,1,10],
    'kernel': ['linear', 'rbf', 'sigmoid'],
    'gamma': ['scale', 'auto']
}

In [42]:

model_4 = GridSearchCV(svc_model, param_grid=param4, cv=5)
model_4.fit(x_train,y_train)

Out[42]:

GridSearchCV(cv=5, estimator=SVC(),
             param_grid={'C': [0.1, 1, 10], 'gamma': ['scale', 'auto'],
                         'kernel': ['linear', 'rbf', 'sigmoid']})

In [43]:

print(model_4.best_params_)
best_svc_model = model_4.best_estimator_

{'C': 1, 'gamma': 'scale', 'kernel': 'linear'}

In [44]:

svcy_pred = best_svc_model.predict(x_test)

In [45]:

accuracy_value.append(accuracy_score(y_test,svcy_pred)*100)

print('Accuracy :', accuracy_score(y_test,svcy_pred)*100)

Accuracy : 80.0

5. Gredient Boosting Classifier Model Training

In [46]:

from sklearn.ensemble import GradientBoostingClassifier
GB_model = GradientBoostingClassifier()

In [47]:

param5 ={
    'learning_rate':[0.1, 0.01, 0.001],
    'n_estimators':[100, 200, 300],
    'max_depth':[3,5,7],
    'min_samples_split':[2,5,10],
    'min_samples_split':[1,2,4]
}

In [48]:

model_5 = GridSearchCV(GB_model, param_grid=param5, cv = 5)
model_5.fit(x_train,y_train)

Out[48]:

GridSearchCV(cv=5, estimator=GradientBoostingClassifier(),
             param_grid={'learning_rate': [0.1, 0.01, 0.001],
                         'max_depth': [3, 5, 7], 'min_samples_split': [1, 2, 4],
                         'n_estimators': [100, 200, 300]})

In [49]:

print(model_5.best_params_)
best_GB_model = model_5.best_estimator_

{'learning_rate': 0.01, 'max_depth': 3, 'min_samples_split': 2, 'n_estimators': 300}

In [50]:

GBy_pred = best_GB_model.predict(x_test)

In [51]:

accuracy_value.append(accuracy_score(y_test,GBy_pred)*100)

print('Accuracy: ', accuracy_score(y_test, GBy_pred)*100)

Accuracy:  81.26984126984127

By using these different type of Machine Learning models, models can be trained and calculate accuracy of them for predicting. Thus, nearly Logistic Regression (79.05%), Decision Tree Classifier (80.63%), Random Forest Classifier (81.58%), Support Vector Classifier (80%) and Gredient Boosting (81.26%) models stands with these accuracy rates.

Instead of only using GridSearchCV() with five cross validation, Stratified K-Fold Spliting Method also can be used with 20 splits.

Using Stratified K-Fold Spliting Method

In [52]:

from sklearn.model_selection import StratifiedKFold, cross_val_score

In [53]:

k_models = []
k_models.append(('Logistic Regression', LogisticRegression()))
k_models.append(('Decision Tree Classifier', DecisionTreeClassifier()))
k_models.append(('Random Forest Classifier', RandomForestClassifier()))
k_models.append(('Support Vector Classifier', SVC()))
k_models.append(('Gredient Boosting',  GradientBoostingClassifier()))

In [54]:

results = []
Names = []
res = []

for  name,model in k_models:
    Kfold = StratifiedKFold(n_splits=20, random_state=None)
    cv_res = cross_val_score(model, x_train, y_train, cv = Kfold, scoring='accuracy')
    results.append(cv_res)
    Names.append(name)
    res.append(cv_res.mean()*100)
    print("%s: %f"% (name, cv_res.mean()*100))

Logistic Regression: 81.320922
Decision Tree Classifier: 75.370124
Random Forest Classifier: 82.382535
Support Vector Classifier: 52.347074
Gredient Boosting: 82.599734

Thus, these 10 type of models' accuracies can be visualized for much understanding as follows.

In [55]:

plt.figure(figsize=(14, 12))
sns.set_theme(style='whitegrid')

plt.subplot(2,1,1)
sns.barplot(x = Names, y = res, palette = "cool")
plt.ylim(50,90)
plt.title('Using GridSearchCV')


Models = ['Logistic Regression' , 'Decision Tree Classifier' , 
          'Random Forest Classifier' , 'Support Vector Classifier', 'Gredient Boosting']
plt.subplot(2,1,2)
sns.barplot(x = Models, y = accuracy_value, palette = 'cool')
plt.title("Using K-Fold Spliting")
plt.ylim(50,90)

plt.show()

Here, the Random Forest Classifier and Gredient Boosting algorithms have shown the highest accuracy values from the trained models. So for the feture predictions, these models can be used for Mental health illness of the persons.

Suggesions and errors are highly appreciated.

-Laksara Abeysinghe