网站设计特点,wordpress站长地图,图书管理系统网站开发教程,wordpress浏览器上主题怎么改名感谢
感谢艾兄#xff08;大佬带队#xff09;、rich师弟#xff08;师弟通过这次比赛机械转码成功、耐心学习#xff09;、张同学#xff08;也很有耐心的在学习#xff09;#xff0c;感谢开源方案#xff08;开源就是银牌#xff09;#xff0c;在此基础上一个月…感谢
感谢艾兄大佬带队、rich师弟师弟通过这次比赛机械转码成功、耐心学习、张同学也很有耐心的在学习感谢开源方案开源就是银牌在此基础上一个月不到收获到了很多运气很好。这个是我们比赛的总结
我们队Kaggle CMI银牌方案欢迎感兴趣的伙伴upvotehttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459610 计划 系统结果稳健取巧
团队计划表每个人做的那部分工作避免重复方便交流提高效率这个工作表起了很大的作用。 具体方案
75th Place Detailed Solution - Spec2DCNN CenterNet Transformer NMS
First of all, I would like to thank tubotubo for sharing your high-quality code, and also thank my teammates liruiqi577 brickcoder xtzhou for their contributions in the competition. Here, I am going to share our team’s “snore like thunder” solution from the following aspects:
Data preprocessingFeature EngineeringModelPost ProcessingModel Ensemble
1. Data preprocessing
We made EDA and readed open discussions found that there are 4 types of data anomalies:
Some series have a high missing rate and some of them do not even have any event labels;In some series , there are no event annotations in the middle and tail (possibly because the collection activity has stopped);The sleep record is incomplete (a period of sleep is only marked with onset or wakeup).There are outliers in the enmo value.
To this end, we have some attempts, such as:
Eliminate series with high missing rates;Cut the tail of the series without event labels;Upper clip enmo to 1.
But the above methods didnt completely work. In the end, our preprocessing method was:
We split the dataset group by series into 5 folds. For each fold, we eliminate series with a label missing rate of 100% in the training dataset while without performing any data preprocessing on the validation set. This is done to avoid introducing noise to the training set, and to ensure that the evaluation results of the validation set are more biased towards the real data distribution, which improve our LB score 0.006.
Part of our experiments as below:
ExperimentFold0Public (single fold)Private (5-fold)No preprocess missing data0.7510.7180.744Eliminate unlabeled data at the end of train_series series with missing rate 80%0.7390.7090.741Drop train series which don’t have any event labels0.7520.7240.749
2. Feature Engineering
Sensor features: After smoothing the enmo and anglez features, a first-order difference is made to obtain the absolute value. Then replace the original enmo and anglez features with these features, which improve our LB score 0.01.
train_series[enmo_abs_diff] train_series[enmo].diff().abs()
train_series[enmo] train_series[enmo_abs_diff].rolling(window5, centerTrue, min_periods1).mean()
train_series[anglez_abs_diff] train_series[anglez].diff().abs()
train_series[anglez] train_series[anglez_abs_diff].rolling(window5, centerTrue, min_periods1).mean()Time features: sin and cos hour.
In addition, we also made the following features based on open notebooks and our EDA, such as: differential features with different orders, rolling window statistical features, interactive features of enmo and anglez (such as anglezs differential abs * enmo, etc.), anglez_rad_sin/cos, dayofweek/is_weekend (I find that children have different sleeping habits on weekdays and weekends). But strangely enough, too much feature engineering didn’t bring us much benefit.
ExperimentFold0Public (5-fold)Private (5-fold)anglez enmo hour_sin hour_cos0.7630.7310.768anglez_abs_diff enmo_abs_diff hour_sin hour_cos0.7710.7410.781
3. Model
We used 4 models:
CNNSpectrogram Spec2DCNN UNet1DDecoder;PANNsFeatureExtractor Spec2DCNN UNet1DDecoder.PANNsFeatureExtractor CenterNet UNet1DDecoder.TransformerAutoModel (xsmall, downsample_rate8).
Parameter Tunning: Add more kernel_size 8 for CNNSpectrogram can gain 0.002 online.
Multi-Task Learning Objectives: sleep status, onset, wake.
Loss Function: For Spec2DCNN and TransformerAutoModel, we use BCE, but with multi-task target weighting, sleep:onset:wake 0.5:1:1. The purpose of this is to allow the model to focus on learning the last two columns. We tried to train only for the onset and wake columns, but the score was not good. The reason is speculated that the positive samples in these two columns are sparse, and MTL needs to be used to transfer the information from positive samples in the sleep status to the prediction of sleep activity events. Also, I tried KL Loss but it didnt work that well.
self.loss_fn nn.BCEWithLogitsLoss(pos_weighttorch.tensor([0.5,1.,1.]))At the same time, we adjusted epoch to 70 and added early stopping with patience15. The early stopping criterion is the AP of the validation dataset, not the loss of the validation set. batch_size32.
ExperimentFold0Public (single fold)Private (5-fold)earlystop by val_loss0.7500.6970.742earlystop by val_score0.7510.7180.744loss_wgt 1:1:10.7520.7240.749loss_wgt 0.5:1:10.7550.7230.753
Note: we used the model_weight.pth with the best offline val_score to submit our LB instead of using the best_model.pth with the best offline val_loss。
4. Post Processing
Our post-processing mainly includes:
find_peaks(): scipy.signal.find_peaks;NMS: This task can be treated as object detection. [onset, wakeup] is regarded as a bounding boxes, and score is the confident of the box. Therefore, I used a time-series NMS. Using NMS can eliminate redundant boxes with high IOU, which increase our AP.
def apply_nms(dets_arr, thresh):x1 dets_arr[:, 0]x2 dets_arr[:, 1]scores dets_arr[:, 2]areas x2 - x1order scores.argsort()[::-1]keep []while order.size 0:i order[0]keep.append(i)xx1 np.maximum(x1[i], x1[order[1:]])xx2 np.minimum(x2[i], x2[order[1:]])inter np.maximum(0.0, xx2 - xx1 1)ovr inter / (areas[i] areas[order[1:]] - inter)inds np.where(ovr thresh)[0]order order[inds 1]dets_nms_arr dets_arr[keep,:]onset_steps dets_nms_arr[:, 0].tolist()wakeup_steps dets_nms_arr[:, 1].tolist()nms_save_steps np.unique(onset_steps wakeup_steps).tolist()return nms_save_stepsIn addition, we set score_th0.005 (If it is set too low, a large number of events will be detected and cause online scoring errors, so it is fixed at 0.005 here), and use optuna to simultaneously search the parameter distance in find_peaks and the parameter iou_threshold of NMS. Finally, when distance72 and iou_threshold0.995, the best performance is achieved.
import optunadef objective(trial):score_th 0.005 # trial.suggest_float(score_th, 0.003, 0.006)distance trial.suggest_int(distance, 20, 80)thresh trial.suggest_float(thresh, 0.75, 1.)# find peakval_pred_df post_process_for_seg(keyskeys,predspreds[:, :, [1, 2]],score_thscore_th,distancedistance,)# nmsval_pred_df val_pred_df.to_pandas()nms_pred_dfs NMS_prediction(val_pred_df, thresh, verboseFalse)score event_detection_ap(valid_event_df.to_pandas(), nms_pred_dfs)return -scorestudy optuna.create_study()
study.optimize(objective, n_trials100)
print(Best hyperparameters: , study.best_params)
print(Best score: , study.best_value)ExperimentFold0Pubic (5-fold)Private (5-fold)find_peak-0.7450.787find_peakNMSoptuna-0.7460.789
5. Model Ensemble
Finally, we average the output probabilities of the following models and then feed into the post processing methods to detect events. By the way, I tried post-processing the detection events for each model and then concating them, but this resulted in too many detections. Even with NMS, I didnt get a better score.
The number of ensemble models: 4 (types of models) * 5 (fold number) 20.
ExperimentFold0Pubic (5-fold)Private (5-fold)model1: CNNSpectrogram Spec2DCNN UNet1DDecoder0.772090.7430.784model2: PANNsFeatureExtractor Spec2DCNN UNet1DDecoder0.7770.7430.782model3: PANNsFeatureExtractor CenterNet UNet1DDecoder0.759680.6340.68model4: TransformerAutoModel0.74680--model1 model2(1:1)-0.7460.789model1 model2model3(1:1:0.4)-0.750.786model1 model2model3model4(1:1:0.4:0.2)0.7520.787
Unfortunately, we only considered CenterNet and Transformer to model ensemble with a tentative attitude on the last day, but surprisingly found that a low-CV-scoring model still has a probability of improving final performance as long as it is heterogeneous compared with your previous models. But we didn’t have more opportunities to submit more, which was a profound lesson for me.
Thoughts not done: Data Augmentation: Shift the time within the batch to increase more time diversity and reduce dependence on hour features. Model: Try more models. Although we try transformer and it didn’t work for us. I am veryyy looking forward to the solutions from top-ranking players.
Thanks again to Kaggle and all Kaggle players. This was a good competition and we learned a lot from it. If you think our solution is useful for you, welcome to upvote and discuss with us.
In addition, this is my first silver medal. Thank you everyone for letting me learn a lot. I will continue to work hard. :)
