PhysioNet/CinC Challenge 2021 Results

The official submissions to this challenge are ranked below, together with the corresponding papers and sources. The all-lead score is computed as the mean of the 12-lead, 3-lead, and 2-lead scores.

Team Name Rank (2-lead) Rank (3-lead) Rank (4-lead) Rank (6-lead) Rank (12-lead) Rank (All-lead) Paper Author(s) Source
ISIBrno-AIMT 1 1 1 1 1 1 Classification of ECG using Ensemble of Residual CNNs with Attention Mechanism Petr Nejedly, Adam Ivora, Ivo Viscor, Zuzana Koscova, Radovan Smisek, Pavel Jurak and Filip Plesinger Link (11MiB)
DSAIL_SNU 2 2 1 1 2 2 Towards High Generalization Performance on Electrocardiogram Classification Hyeongrok Han, Seongjae Park, Seonwoo Min, Hyun-Soo Choi, Eunji Kim, Hyunki Kim, Sangha Park, Jin-Kook Kim, Junsang Park, Junho An, Kwnanglo Lee, Wonsun Jeong, Sangil Chon, Kwonwoo Ha, Myungkyu Han and Sungroh Yoon Link (27MiB)
NIMA 3 3 3 5 2 3 Multi-label Cardiac Abnormality Classification from Electrocardiogram using Deep Convolutional Neural Networks Nima L Wickramasinghe and Mohamed Athif Link (13KiB)
cardiochallenger 4 5 4 5 2 4 Channel Self-Attention Deep Learning Framework for Multi-Cardiac Abnormality Diagnosis from Varied-Lead ECG Signals Apoorva Srivastava, Ajith Hari, Sawon Pratiher, sazedul alam, Nirmalya Ghosh, Nilanjan Banerjee and Amit Patra Link (21KiB)
CeZIS 5 4 5 3 6 5 A Two-Phase Multilabel ECG Classification Using One-Dimensional Convolutional Neural Network and Modified Labels Peter Bugata, Peter Bugata Jr., Vladimira Kmecova, Monika Stankova, David Gajdos, David Hudak, Richard Stana, Simon Horvat, Lubomir Antoni, Gabriela Vozarikova, Erik Bruoth and Alexander Szabari Link (260MiB)
DataLA_NUS 5 8 6 5 8 7 Robust and Task-Aware Training of Deep Residual Networks for Varying-Lead ECG Classification Hansheng Ren, Miao Xiong and Bryan Hooi Link (681KiB)
USST_Med 7 5 7 3 5 5 Classifying Different Dimensional ECGs Using Deep Residual Convolutional Neural Networks Wenjie Cai, Fanli Liu, Xuan Wang, Bolin Xu, Yaohui Wang Link (18MiB)
SMS+1 8 8 7 16 6 7 Two will do: Convolutional Neural Network with Asymmetric Loss and Self-Learning Label Correction for Imbalanced Multi-Label ECG Data Classification Cristina Gallego Vázquez, Alexander Breuss, Oriella Gnarra, Julian Portmann and Giulia Da Poian Link (52KiB)
Dr_Cubic 9 5 7 8 9 9 Towards Generalization of Cardiac Abnormality Classification Using ECG Signal Xiaoyu Li, Chen Li, Xian Xu, Yuhua Wei, Jishang Wei, Yuyao Sun, Buyue Qian and Xiao Xu Link (75KiB)
ami_kagoshima 10 11 10 11 9 10 Reduced-Lead ECG Classifier Model Trained with DivideMix and Model Ensemble Hiroshi Seki, Takashi Nakano, Koshiro Ikeda, Shinji Hirooka, Takaaki Kawasaki, Mitsutomo Yamada, Shumpei Saito, Toshitaka Yamakawa, Shimpei Ogawaamikagoshima Link (45KiB)
BUTTeam 11 15 11 14 14 15 Cardiac Abnormalities Recognition in ECG Using a Convolutional Network with Attention and Input with an Adaptable Number of Leads Tomas Vicar, Petra Novotna, Jakub Hejc, Oto Janousek and Marina Ronzhina Link (500KiB)
iadi-ecg 11 11 11 11 12 11 Cardiac Abnormality Detection based on an Ensemble Voting of Single-Lead Classifier Predictions Pierre Aublin, Julien Oster, Mouin Ben Ammar, Jérémy Fix and Michel Barret Link (59KiB)
Polimi_1 13 10 15 14 14 11 Ensemble Learning of Modified Residual Networks for Classifying ECG with Different Set of Leads Federico M Muscato, Valentina D A Corino and Luca T Mainardi Link (20KiB)
snu_adsl 13 11 11 10 12 11 Learning ECG Representations for Multi-Label Classification of Cardiac Abnormalities Jangwon Suh, Jimyeong Kim, Eunjung Lee, Jaeill Kim, Duhun Hwang, Jungwon Park, Junghoon Lee, Jaeseung Park, Seo-Yoon Moon, Yeonsu Kim, Min Kang, Soonil Kwon, Eue-Keun Choi and Wonjong Rhee Link (736KiB)
prna 15 11 14 9 9 11 Convolution-Free Waveform Transformers for Multi-Lead ECG Classification Annamalai Natarajan, Gregory Boverman, Yale Chang, Corneliu Antonescu and Jonathan Rubin Link (4.6GiB)
ibmtPeakyFinders 16 18 23 11 14 16 Automatic Classification of 12-, 6-, 4-, 3-, and 2-Lead Electrocardiograms Using Morphological Feature Extraction Alexander Hammer, Matthieu Scherpf, Hannes Ernst, Jonas Weiß, Daniel Schwensow and Martin Schmidt Link (5.4MiB)
UMCU 16 17 17 18 14 16 Automated Diagnosis of Reduced-lead Electrocardiograms using a Shared Classifier Hidde Jessen, Rutger van de Leur, Pieter Doevendans and Rene van Es Link (35MiB)
Gio_new_img 18 18 18 18 18 18 3-D ECG images with Deep Learning Approach for Identification of Cardiac Abnormalities from a Variable Number of Leads giovanni bortolan Link (215MiB)
csu_anying 19 21 16 17 25 21 MTFNet: A Morphological and Temporal Features Network for multiple leads ECG Classification Lebing Pan, Weibai Pan, Mengxue Li, Yuxia Guan and Ying An Link (12MiB)
AIRCAS_MEL1 20 16 20 23 21 18 A Novel Multi-Scale Convolutional Neural Network for Arrhythmia Classification on Reduced-lead ECGs Pan Xia, Zhengling He, Yusi Zhu, Zhongrui Bai, Xianya Yu, Yuqi Wang, Fanglin Geng, Lidong Du, Xianxiang Chen, Peng Wang, Zhen Fang Link (20KiB)
METU-19 20 20 19 20 21 20 Two Might Do: A Beat-by-Beat Classification of Cardiac Abnormalities using Deep Learning with Domain-Specific Features Berken Utku Demirel, Adnan Harun Dogan and Mohammad Abdullah Al Faruque Link (18KiB)
Biomedic2ai 22 27 26 27 23 24 Detecting Cardiac Abnormalities with Multi-Lead ECG Signals: A Modular Network Approach Ryan Clark, Mohammadreza Heydarian, Kashif Siddiqui, Sajjad Rashidiani, Md Asif Khan and Thomas E Doyle Link (15KiB)
HaoWan_AIeC 22 23 20 23 18 22 A Mixed-Domain Self-Attention Network for Multilabel Cardiac Irregularity Classification Using Reduced-Lead Electrocardiogram Hao-Chun Yang, Wan-Ting Hsieh and Trista Pei-Chun Chen Link (597KiB)
itaca-UPV 22 25 29 21 26 24 Multiple Cardiac Disease Detection from Minimal-Lead ECG Combining Feedforward Neural Networks with a One-vs-Rest Approach Santiago Jiménez-Serrano, Miguel Rodrigo, Conrado J. Calvo, José Millet Roig and Francisco Castells Link (46KiB)
UoB_HBC 22 22 22 21 23 23 An InceptionTime-Inspired Convolutional Neural Network to Detect Cardiac Abnormalities in Reduced-Lead ECG Data Harry Crocker and Aaron Costall Link (29KiB)
Eagles 26 25 27 27 27 27 Reduced-Lead Electrocardiogram Classification using Wavelet Analysis and Deep Learning Adrian K Cornely, Alondra Carrillo and Grace Mirsky Link (14KiB)
HeartlyAI 27 23 23 25 18 24 Rethinking ECG Classification with Neural Networks as a Sequence-to-Sequence Task Philipp Sodmann, Marcus Vollmer and Lars Kaderali Link (174MiB)
AADAConglomerate 28 30 31 30 31 30 Multi-Label Classification of Cardiac Abnormalities for Multi-Lead ECG Recordings Based on Auto-Encoder Features and a Neural Network Classifier Onno Linschmann, Maurice Rohr, Klaus Steffen Leonhardt1 and Christoph Hoog Antink Link (49KiB)
BiSP_Lab 28 29 28 29 27 28 Link (16KiB) Classification of ECG Signals with Different Lead Systems Using AutoML Matteo Bodini, Massimo W Rivolta and Roberto Sassi
DSC 28 28 25 25 29 28 Pathologies Prediction on Short ECG Signals with Focus on Feature Extraction Based on Beat Morphology and Image Deformation Jeffrey Prehn, Svetoslav Ivanov and Georgi Nalbantov Link (923KiB)
Care4MyHeart 31 31 30 30 30 31 ResNet-BiLSTM Network Activations Mohanad Alkhodari, Georgios Apostolidis, Charilaos Zisou, Leontios Hadjileontiadis and Ahsan Khandoker Link (175KiB)
Sunset 31 32 32 32 32 32 Using Mel-frequency Cepstrum and Amplitude-time Heart Variability as XGBoost Handcrafted Features for Heart Disease Detection Sergey Krivenko, Anatolii Pulavskyi, Liudmyla Kryvenko, Olha Krylova and Stanislav Krivenko Link (78KiB)
CardioIQ 33 33 33 33 33 33 An Ensemble Learning Approach to Detect Cardiac Abnormalities in ECG Data Irrespective of Lead Availability Tim Uhlemann, Sebastian Wegener, Joshua Prim, Nils Gumpfer, Dimitri Grün, Jennifer Hannig, Till Keller and Michael Guckert Link (16KiB)
BitScattered 34 34 34 33 34 34 Arrhythmia Classification of Reduced-Lead Electrocardiograms by Scattering-Recurrent Networks Philip A. Warrick, Vincent Lostanlen, Michael Eickenberg, Masun Nabhan Homsi, Adrian Campoy Rodriguez and Joakim Anden Link (15MiB)
Cordi-Ak 35 35 35 35 35 35 Leveraging Period-specific Variations in ECG Topology for Classification Tasks Paul Samuel Ignacio Link (28KiB)
heartMAASters 36 36 36 36 37 36 Multi-Label Classification on 12, 6, 4, 3 and 2 Lead Electrocardiography Signals Using Convolutional Recurrent Neural Networks Niels Osnabrugge, Kata Keresztesi, Felix Rustemeyer, Christos Kaparakis, Francesca Battipaglia, Pietro Bonizzi and Joel Karel Link (32KiB)
easyG 37 37 37 37 36 37 ECG Classification Combining Conventional Signal Analysis, Random Forests and Neural Networks – a Stacked Learning Scheme Martin Baumgartner, Martin Kropf, Lukas Haider, Sai Veeranki, Dieter Hayn and Günter Schreier Link (1.3GiB)
WEAIT 38 39 37 38 38 38 N-BEATS for Heart Disfunction Classification Bartosz Puszkarski, Krzysztof Hryniów, and Grzegorz Sarwas Link (22KiB)
CardiOUS 39 38 39 39 39 38 Multi-label ECG classification using Convolutional Neural Networks in a Classifier Chain Bjørn-Jostein Singstad, Eraraya Muten and Pål Brekke Link (74KiB)

Unofficial entries

AI_Healthcare Deep Discriminative Domain Generalization with Adversarial Feature Learning for Classifying ECG Signals Zuogang Shang, Zhibin Zhao, Hui Fang, Samuel Relton, Darcy Murphy, Zoe Hancox, Ruqiang Yan and David Wong Link (138MiB)  
AIHealth     Link (2.4MiB)  
bric     Link (124MiB)  
CardioLux     Link (33KiB)  
CinCSEM Generative Pre-Trained Transformer for Cardiac Abnormality Detection Pierre Louis Gaudilliere, Halla Sigurthorsdottir, Clémentine Aguet, Jérôme Van Zaen, Mathieu Lemay and Ricard Delgado-Gonzalo Link (67KiB)  
ConnectedHealth     Link (25KiB)  
CQUPT_Dontwant A Branched Deep Neural Network for End-to-end Classification from ECGs with Varying Dimensions Han Duan, Junchao Fan, Junhui Zhang, Bin Xiao, Xiuli Bi and Xu Ma    
GenieEnterprise     Link (65KiB)  
HeartBeats Multi-Label Cardiac Abnormalities Classification on Selected Leads of ECG Signals Zhuoyang Xu, Yangming Guo, Tingting Zhao, Zhuo Liu and Xingzhi Sun Link (1.5GiB)  
Leicester-Fox Spatio-temporal ECG Network for Detecting Cardiac Disorders from Multi-lead ECGs Long Chen, Zheheng Jiang, Tiago P. Almeida, Fernando S. Schlindwein, Jakevir S. Shoker, G. Andre Ng, Huiyu Zhou and Xin Li    
LINC Automatic Diagnosis of Cardiac Disease from Twelve-lead and Reduced-lead ECGs using Multi-label Classification Prathic Sundararajan, Kevin Moses, Cristhian Potes and Saman Parvaneh    
matFCT Semi-supervised Learning for ECG Classification Rui Rodrigues and Paula Couto Link (97MiB)  
Medics Diagnosis of Cardiac Abnormalities Applying Scattering Transform and Fourier-Bessel Expansion on ECG Signals Nidhi Kalidas Sawant and Shivnarayan Patidar    
PhysioNauts Combining a ResNet Model with Handcrafted Temporal Features for ECG Classification with Varying Number of Leads Stefano Magni, Andrea Sansonetti, Chiara Salvi, Tiziana Tabiadon and Guadalupe García Isla Link (43KiB)  
Proton Multi-Label Classification of Multi-lead ECG Based on Deep 1D Convolutional Neural Networks With Residual and Attention Mechanism Yamin Liu, Hanshuang Xie, Qineng Cao, Jiayi Yan, Fan Wu, Huaiyu Zhu and Yun Pan Link (60MiB)  
Revenger Hybrid Arrhythmia Detection on Varying-Dimensional Electrocardiography: Combining Deep Neural Networks and Clinical Rules Hao Wen and Jingsu Kang Link (105MiB)  
SDSVLS     Link (37KiB)  
SharifHeart     Link (352MiB)  
skylark Incorporating Demographic and Heartbeat Features with Multichannel ECG for Cardiac Abnormality Detection using Parallel CNN and GAP Network Deepankar Nankani and Rashmi Dutta Baruah Link (42MiB)  
TeamLeo     Link (17KiB)  
UIDT_UNAM Cardiac Anomalies Detection Through 2D-CNN and ECG Spectrograms Jonathan Roberto Torres Castillo and Miguel Padilla Castañeda Link (20KiB)  
[Unknown name]     Link (261MiB)  

Scores

These tables contain scores for the 12-lead, 6-lead, 4-lead, 3-lead, and 2-lead versions of the hidden validation and test sets. The 2021 Challenge webpage and 2021 Challenge paper describes the lead combinations and the validation and test data sources. We also included an all-lead score in the tables, which is computed as the mean of the 12-lead, 3-lead, and 2-lead scores.

We used the Challenge scoring metric that we implemented in the evaluation code repository to rank the official entries on the test set. We sorted the unofficial entries alphabetically by team name.

In these tables, you can find the following information:

To refer to these tables in your publication, please cite the following papers:

Reyna MA, Sadr N, Perez Alday EA, Gu A, Shah AJ, Robichaux C, Rad AB, Elola A, Seyedi S, Ansari S, Ghanbari H, Li Q, Sharma A, Clifford GD. Will Two Do? Varying Dimensions in Electrocardiography: The PhysioNet/Computing in Cardiology Challenge 2021. Computing in Cardiology 2021; 48: 1-4

Reyna MA, Sadr N, Perez Alday EA, Gu Annie, Shah AJ, Robichaux C, Rad AB, Elola A, Seyedi S, Ansari S, Ghanbari H, Li Q, Sharma A, Clifford GD. Issues in the automated classification of multilead ECGs using heterogeneous labels and populations. Physiol. Meas. 2022; 43(8): 084001

Perez Alday EA, Rad AB, Reyna MA, Sadr N, Gu A, Li Q, Dumitru M, Xue J, Albert D, Sameni R, Clifford GD. Age, sex and race bias in automated arrhythmia detectors. Journal of Electrocardiology 2022 July 22; 74: pp. 5-9. DOI: 10.1016/j.jelectrocard.2022.07.007

Supported by the National Institute of Biomedical Imaging and Bioengineering (NIBIB) under NIH grant number R01EB030362.

Back