Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020

Papers, code, and scores for the 2020 Challenge are available on this page.

Citations

To refer to the 2020 Challenge, please cite:

Perez Alday EA, Gu A, J Shah A, Robichaux C, Ian Wong AK, Liu C, Liu F, Bahrami Rad A, Elola A, Seyedi S, Li Q, Sharma A, Clifford GD* Reyna MA*. Classification of 12-lead ECGs: The PhysioNet/Computing in Cardiology Challenge 2020. Physiol. Meas. 2021 Jan 1;41(12):124003. doi: 10.1088/1361-6579/abc960 to refer to the 2020 Challenge.

Please also cite the standard PhysioNet citation. You can find followup articles to the 2020 Challenge in the 2021 Challenge and in the Journal of Physiological Measurement Focus Issue on Classification of Multilead ECGs.

Challenge Results

The conference papers for Computing in Cardiology 2020 are available on the CinC and IEEE websites.

The ranks of the teams and their papers and entry code are available in the table below.

Rank Team name Team member(s) Paper Code
1 prna Annamalai Natarajan, Yale Chang, Sara Mariani, Asif Rahman, Gregory Boverman, Shruti Vij and Jonathan Rubin A Wide and Deep Transformer Neural Network for 12-Lead ECG Classification Link (86MB)
2 Between a ROC and a heart place Zhibin Zhao, Hui Fang, Samuel Relton, Ruqiang Yan, Yuhong Liu, Zhijing Li, Jing Qin and David Wong Adaptive lead weighted ResNet trained with different duration signals for classifying 12-lead ECGs Link (30MB)
3 HeartBeats Zhaowei Zhu, Han Wang, Tingting Zhao, Yangming Guo, Zhuoyang Xu, Zhuo Liu, Siqi Liu, Xiang Lan, Xingzhi Sun and Mengling Feng Classification of Cardiac Abnormalities From ECG Signals Using SE-ResNet Link (145MB)
4 Triage Maximilian Oppelt, Maximilian Riehl, Felix Kemeth and Jan Steffan Combining Scatter Transform and Deep Neural Networks for Multilabel ECG Signal Classification Link (79MB)
5 Sharif AI Team Hosein Hasani, Adeleh Bitarafan and Mahdieh Soleymani Classification of 12-lead ECG Signals with Adversarial Multi-Source Domain Generalization Link (25MB)
6 DSAIL_SNU Seonwoo Min, Hyun-Soo Choi, Hyeongrok Han, Minji Seo, Jin-Kook Kim, Junsang Park, Sunghoon Jung, Il-Young Oh, Byunghan Lee and Sungroh Yoon Bag of Tricks for Electrocardiogram Classification with Deep Neural Networks Link (99MB)
7 UMCUVA Max Bos, Rutger van de Leur, Jeroen Vranken, Deepak Gupta, Pim van der Harst, Pieter Doevendans and René van Es Automated Comprehensive Interpretation of 12-lead Electrocardiograms Using Pre-trained Exponentially Dilated Causal Convolutional Neural Networks Link (286MB)
8 CQUPT_ECG Jiabo Chen, Tianlong Chen, Bin Xiao, Xiuli Bi, Yongchao Wang, Weisheng Li, Han Duan, Junhui Zhang and Xu Ma SE-ECGNet: Multi-scale SE-Net for Multi-lead ECG Data Link (221KB)
9 ECU Najmeh Fayyazifar, Selam Ahderom, David Suter, Andrew Maiorana and Girish dwivedi Impact of Neural Architecture Design on Cardiac Abnormality Classification Using 12-lead ECG Signals Link (151KB)
10 PALab wenxiao jia, Xiao Xu, Xian Xu, Yuyao Sun and Xiaoshuang Liu Arrhythmia Detection and Classification of 12-lead ECGs Using a Deep Neural Network Link (402KB)
11 HITTING Radovan Smisek, Andrea Nemcova, Lucie Marsanova, Lukas Smital, Martin Vítek and Jiri Kozumplik Cardiac Pathologies Detection and Classification in 12-lead ECG Link (58MB)
12 Gio_Ivo giovanni bortolan, Ivaylo Christov and Iana Simova Rule-Based methods and Deep Learning Networks for Automatic Classification of ECG Link (87MB)
13 AUTh Team Charilaos Zisou, Andreas Sochopoulos and Konstantinos Kitsios Convolutional Recurrent Neural Network and LightGBM Ensemble Model for 12-lead ECG Classification Link (52MB)
14 BioS Mateusz Soliński, Michał Łepek, Antonina Pater, Katarzyna Muter, Przemysław Wiszniewski, Dorota Kokosińska, Judyta Salamon and Zuzanna Puzio 12-lead ECG Arrythmia Classification Using Convolutional Neural Network for Mutually Non-Exclusive Classes Link (58MB)
15 UC_Lab_Kn Lucas Weber, Maksym Gaiduk, Wilhelm Daniel Scherz and Ralf Seepold Cardiac Abnormality Detection in 12-lead ECGs with Deep Convolutional Neural Networks Using Data Augmentation Link (208KB)
16 Cardio-Challengers Akash Kirodiwal, Apoorva Srivastava, Ashutosh Dash, Ayantika Saha, Gopi Vamsi Penaganti, Sawon Pratiher, sazedul alam, Amit Patra, Nirmalya Ghosh and Nilanjan Banerjee A Bio-toolkit for Multi-Cardiac Abnormality Diagnosis Using ECG Signal and Deep Learning Link (987KB)
17 JuJuRock Po-Ya Hsu, Po-Han Hsu, Tsung-Han Lee and Hsin-Li Liu Multi-label Arrhythmia Classification From 12-Lead Electrocardiograms Link (36MB)
18 Minibus Ran Duan, Xiaodong He and Ouyang Zhuoran MADNN: A Multi-scale Attention Deep Neural Network for Arrythmia Classification Link (17MB)
19 Desafinado Durmus Umutcan Uguz, Felix Berief, Steffen Leonhardt and Christoph Hoog Antink Classification of 12-lead ECGs Using Gradient Boosting on Features Acquired With Domain-Specific and Domain-Agnostic Methods Link (255MB)
20 Team UIO Bjørn-Jostein Singstad and Christian Tronstad Convolutional Neural Network and Rule-Based Algorithms for Classifying 12-lead ECGs Link (19KB)
21 Eagles Andrew Demonbreun and Grace Mirsky Automated Classification of Electrocardiograms Using Wavelet Analysis and Deep Learning Link (60MB)
22 BUTTeam Tomas Vicar, Jakub Hejc, Petra Novotna, Marina Ronzhina and Oto Janousek ECG Abnormalities Recognition Using Convolutional Network With Global Skip Connections and Custom Loss Function Link (124MB)
23 DSC Georgi Nalbantov, Svetoslav Ivanov and Jeffrey van Prehn Multi-Class Classification of Pathologies Found on Short ECG Signals Link (5.4MB)
24 Pink Irish Hat Halla Sigurthorsdottir, Jérôme Van Zaen, Ricard Delgado-Gonzalo and Mathieu Lemay ECG Classification With a Convolutional Recurrent Neural Network Link (7.6MB)
25 Madhardmax Hardik Rajpal, Madalina Sas, Rebecca Joakim, Chris Lockwood, Nicholas S. Peters and Max Falkenberg Interpretable XGBoost Based Classification of 12-lead ECGs Applying Information Theory Measures From Neuroscience Link (7.0MB)
26 Care4MyHeart Mohanad Alkhodari, Leontios J. Hadjileontiadis and Ahsan H. Khandoker Identification of Cardiac Arrhythmias from 12-lead ECG using Beat-wise Analysis and a Combination of CNN and LSTM Link (43MB)
27 MCIRCC Sardar Ansari, Christopher Gillies, Brandon Cummings, Jonathan Motyka, Guan Wang, Kevin Ward and Hamid Ghanbari Classification of 12-Lead Electrocardiograms Using Residual Neural Networks and Transfer Learning Link (30MB)
28 heartly-ai Philipp Sodmann and Marcus Vollmer ECG Segmentation using a Neural Network as the Basis for Detection of Cardiac Pathologies Link (42MB)
29 Code Team Antonio H. Ribeiro, Daniel Gedon, Daniel Martins Teixeira, Manoel Horta Ribeiro, Antonio Luiz Ribeiro, Thomas B. Schön and Wagner Meira Jr Automatic 12-lead ECG Classification Using a Convolutional Network Ensemble Link (276KB)
30 ISIBrno Petr Nejedly, Adam Ivora, Ivo Viscor, Josef Halamek, Pavel Jurak and Filip Plesinger Utilization of Residual CNN-GRU with Attention Mechanism for Classification of 12-lead ECG Link (248KB)
31 Alba_W.O. Marek Żyliński and Gerard Cybulski Selected Features for Classification of 12-lead ECGs Link (58MB)
32 AI Strollers Rohit Pardasani and Navchetan Awasthi Classification of 12 Lead ECG Signal Using 1D-CNN With Class Dependent Threshold Link (41MB)
33 ECGLearner Yingjing Feng and Edward Vigmond Deep Multi-Label Multi-Instance Classification on 12-Lead ECG Link (413KB)
34 Leicester-Fox Zheheng Jiang, Tiago Paggi de Almeida, Fernando Schlindwein, G. André Ng, Huiyu Zhou and Xin Li Diagnostic of Multiple Cardiac Disorders from 12-lead ECGs Using Graph Convolutional Network Based Multi-label Classification Link (39MB)
35 deepzx987 Deepankar Nankani, Pallabi Saikia and Rashmi Dutta Baruah Automatic Concurrent Arrhythmia Classification Using Deep Residual Neural Networks Link (555KB)
36 CVC Alexander William Wong, Weijie Sun, Sunil Vasu Kalmady, Padma Kaul and Abram Hindle Multilabel 12-Lead Electrocardiogram Classification Using Gradient Boosting Tree Ensemble Link (12MB)
37 Cordi-Ak Paul Samuel Ignacio, Jay-Anne Bulauan and John Rick Manzanares A Topology Informed Random Forest Classifier for ECG Classification Link (891KB)
38 MIndS Marwen Sallem, Adnen Saadaoui, Amina Ghrissi and Vicente Zarzoso Detection of Cardiac Arrhythmias From Varied Length Multichannel Electrocardiogram Recordings Using Deep Convolutional Neural Networks Link (47MB)
39 easyG Martin Baumgartner, Dieter Hayn, Andreas Ziegl, Alphons Eggerth and Günter Schreier Multi-Stream Deep Neural Network for 12-Lead ECG Classification Link (399KB)
40 BiSP Lab Matteo Bodini, Massimo W Rivolta and Roberto Sassi Classification of 12-lead ECG with an Ensemble Machine Learning Approach Link (340KB)
41 Technion_AIMLAB David Assaraf, Jeremy Levy, Janmajay Singh, Armand Chocron and Joachim A. Behar Classification of 12-lead ECGs using digital biomarkers and representation learning Link (474KB)

Unranked submissions:

Team name Team member(s) Paper Code
AAIST Link (8.3KB)
AImsterdam Link (27MB)
BERCLAB UND Link (12MB)
BME_Feng Link (7.3MB)
BraveHeart400 Link (147MB)
BRIC Link (276KB)
Chapman Link (24KB)
Connected_Health Link (29KB)
Health team Szeged Link (19KB)
IBMTpeakyFinders Link (9.0MB)
Kimball_IRL Link (27KB)
LaussenLabs Sebastian Goodfellow, Dmitrii Shubin, Danny Eytan, Andrew Goodwin, Anusha Jega, Azadeh Assadi, Mjaye Mazwi, Robert Greer, Sujay Nagaraj, Peter Laussen, William Dixon and Carson McLean Rhythm classification of 12-lead ECGs using deep neural network and class-activation maps for improved explainability Link (13MB)
LIST_AIHealthCare Link (26KB)
Marquette David Kaftan and Richard Povinelli A Deep Neural Network and Reconstructed Phase Space Approach to Classifying 12-lead ECGs Link (21MB)
Medics Link (58KB)
MetaHeart Zhengling He, Pengfei Zhang, Lirui Xu, Zhongrui Bai, Hao Zhang, Weisong Li, Pan Xia and Xianxiang Chen A Novel Convolutional Neural Network for Arrhythmia Detection From 12-lead Electrocardiograms Link (25MB)
Metformin-121 Link (16KB)
ML Warriors Link (3.6MB)
NACAS_12X Link (100MB)
nebula Wenjie Cai, Shuaicong Hu, Jingying Yang and Jianjian Cao Automatic 12-lead ECG Classification Using Deep Neural Networks Link (45MB)
NN-MIH Naoki Nonaka and Jun Seita Electrocardiogram Classification by Modified EfficientNet with Data Augmentation Link (59MB)
NTU-Accesslab Yu-Cheng Lin, Yun-Chieh Lee, Wen-Chiao Tsai, Win-Ken Beh and An-Yeu Wu Explainable Deep Neural Network for Identifying Cardiac Abnormalities Using Class Activation Map Link (339KB)
Orange Peel Link (12KB)
SBU_AI Ibrahim Hammoud, IV Ramakrishnan and Petar Djuric Classification of 12-lead ECGs Using Intra-Heartbeat Discrete-time Fourier Transform and Inter-Heartbeat Attention Link (13MB)
SpaceOn Flattop Shan Yang, Heng Xiang, Qingda Kong and Chunli Wang Multi-label Classification of Electrocardiogram With Modified Residual Networks Link (364MB)
try again Link (11KB)
UniA4Life Link (80KB)

Tables of final scores

The final scores for the 2020 PhysioNet/CinC Challenge are in the following tables:

  1. The official scores
  2. The unofficial scores
  3. The metrics per database of the official entries
  4. The per-class scoring metrics of the official entries on the validation data

We introduced a new scoring metric for this Challenge. We used this scoring metric to evaluate and rank the Challenge entries. We included several other metrics for reference. The area under the receiver operating characteristic (AUROC), area under the precision recall curve (AUPRC), and F-measure scores are the macro-average of the scores across all classes. The accuracy metric is the fraction of correctly diagnosed recordings, i.e., all classes for the recording are correct. These metrics were computed by the evaluate_12ECG_score.py script. Please see the script for more details of these scores.

We included the scores on the following datasets:

  1. Validation Set: Includes recordings from the hidden CPSC and G12EC sets.
  2. Hidden CPSC Set: Split between the validation and test sets.
  3. Hidden G12EC Set: Split between the validation and test sets.
  4. Hidden Undisclosed Set: All recordings were part of the test sets.
  5. Test Set: Includes recordings from the hidden CPSC, G12EC, and undisclosed test sets.

To refer to these tables in a publication, please cite Perez Alday EA, Gu A, Shah AJ, Robichaux C, Wong AI, Liu C, Liu F, Rad AB, Elola A, Seyedi S, Li Q, Sharma A, Clifford GD*, Reyna MA*. Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020. Physiol Meas. 41 (2020). doi: 10.1088/1361-6579/abc960.

In these tables, you can find the following information:

  1. Official entries that were scored on the validation and test data and ranked in the Challenge: physionet_2020_official_scores.csv
  2. Unofficial entries that were scored on the validation and test data but unranked because they did not satisfy all of the rules or were unsuccessful on one or more of the test sets: physionet_2020_unofficial_scores.csv
  3. Challenge and other scoring metrics on all official entries broken with scores for each database in the validation and test data: physionet_2020_metrics_perDatabase_official_entries.csv
  4. Per-class scoring metrics on the validation data: physionet_2020_validation_metrics_by_class_official_entries.csv

Supported by the National Institute of Biomedical Imaging and Bioengineering (NIBIB) under NIH grant number R01EB030362.

Back