Below is a ranking of all the CASP12 contact predictors in terms of the total F1 score on 38 free modeling targets

when top L/5 long- and medium-range contacts are evaluated. Ranking does not change much when top L/2 contacts are evaluated.

The CASP12 assessor also proposed an incorrect ranking metric F1(prob),

but then abandoned it. Here is the explanation why F1(prob) is wrong.

 

Group Name

Rank

F1

Precision

Recall

RaptorX-Contact   

1

12.386

55.831

7.029

MetaPSICOV        

2

10.919

51.307

6.155

iFold_1           

3

10.917

50.916

6.153

MULTICOM-CONSTRUCT

4

10.847

50.449

6.121

Pcons-net         

5

10.810

49.536

6.113

RBO-Epsilon       

6

10.736

48.601

6.081

FALCON_COLORS     

7

10.387

47.253

5.880

Yang-Server       

8

10.186

46.460

5.763

Deepfold-Contact  

9

10.003

46.442

5.644

PconsC31           

10

 9.734

45.728

5.483

IGBteam           

11

 9.428

45.596

5.288

MULTICOM-CLUSTER  

12

 9.114

42.661

5.138

naive             

13

 9.016

42.009

5.085

raghavagps        

14

 9.005

40.269

5.110

Shen-Group        

15

 8.984

40.938

5.085

AkbAR             

16

 8.901

41.417

5.021

MULTICOM-NOVEL    

17

 8.235

36.630

4.680

Zhang_Contact     

18

 8.111

38.259

4.570

PLCT              

19

 7.690

35.133

4.356

PconsC2           

20

 7.347

34.976

4.132

Distill           

21

 5.889

27.762

3.315

FLOUDAS_SERVER    

22

 5.371

24.255

3.041

BG2               

23

 5.073

25.475

2.831

BAKER_GREMLIN     

24

 5.072

25.451

2.831

myprotein-me      

25

 4.442

21.699

2.485

ZHOU-SPARKS-X     

26

 3.982

19.030

2.236

Wang4             

27

 3.547

17.193

1.988

Wang2             

28

 2.664

11.725

1.517

FLOUDAS            

29

 2.566

11.617

1.452

RRCpred           

30

 2.196

10.051

1.243

Wang3             

31

 2.163

 9.359

1.235

Wang1             

32

 1.825

 7.733

1.046

KScons            

33

 0.505

 2.521

0.281

FONT

34

 0.160

 0.857

0.088