NO.PZ2021083101000016
问题如下：
Achler splits the DTM into training, cross-validation, and test datasets. Achler uses a supervised learning approach to train the logistic regression model in predicting sentiment. Applying the receiver operating characteristics (ROC) technique and area under the curve (AUC) metrics, Achler evaluates model performance on both the training and the cross-validation datasets. The trained model performance for three different logistic regressions’ threshold p-values is presented in Exhibit 3.
Based on Exhibit 3, which threshold p-value indicates the best fitting model?
选项：
A.
0.57
B.
0.79
C.
0.84
解释：
B is correct. The higher the AUC, the better the model performance. For the threshold p-value of 0.79, the AUC is 91.3% on the training dataset and 89.7% on the cross- validation dataset, and the ROC curves are similar for model performance on both datasets. These findings suggest that the model performs similarly on both training and CV data and thus indicate a good fitting model.
A is incorrect because for the threshold p-value of 0.57, the AUC is 56.7% on the training dataset and 57.3% on the cross- validation dataset. The AUC close to 50% signifies random guessing on both the training dataset and the crossvalidation dataset. The implication is that for the threshold p-value of 0.57, the model is randomly guessing and is not performing well.
C is incorrect because for the threshold p-value of 0.84, there is a substantial difference between the AUC on the training dataset (98.4%) and the AUC on the cross- validation dataset (87.1%). This suggests that the model performs comparatively poorly (with a higher rate of error or misclassification) on the cross- validation dataset when compared with training data. Thus, the implication is that the model is overfitted.
考点：Model Training: Performance Evaluation
就是不知道那个P-value有什么用处，还有这两个概率相加到什么程度才过度拟合和，没明白。
