Note: this is no longer a wiki, only a static archive of the orginal!

Home

Top Three Scores

The following tables give the top three scores for each data set in the CoNLL 2007 shared task, as well as the average score, the standard deviation, and the number of teams that have submitted results for that data set. The "Multilingual_average" row contains your average score for all ten languages in the multilingual track, which determines your ranking. The ranking for the domain adaptation track is determined based on the pchemtb results only. The CHILDES data set is optional.

LABELED ATTACHMENT SCORE

Task

1st

2nd

3rd

Avg

Std

Nteams

Arabic

76.52

75.08

74.75

68.34

07.92

20

Basque

76.94

75.75

75.49

68.07

08.02

20

Catalan

88.70

88.16

87.90

79.85

14.88

20

Chinese

84.69

83.84

83.51

76.59

09.16

21

Czech

80.19

78.60

77.98

70.12

09.19

20

English

89.61

89.01

88.41

80.95

10.26

23

Greek

76.31

74.65

74.42

70.22

06.41

20

Hungarian

80.27

79.53

78.09

71.49

08.60

21

Italian

84.40

83.91

83.61

78.06

06.71

20

Turkish

79.81

79.79

79.24

73.19

06.77

20

Multilingual_average

80.32

80.29

79.90

65.77

22.17

23

pchemtb_closed

81.06

80.40

80.22

73.03

10.74

8

pchemtb_open

78.48

63.49

63.01

65.11

09.64

4

UNLABELED ATTACHMENT SCORE

Task

1st

2nd

3rd

Avg

Std

Nteams

Arabic

86.09

85.81

84.21

78.84

07.07

20

Basque

82.84

81.93

81.19

75.18

07.20

20

Catalan

93.40

93.34

93.12

87.98

06.55

20

Chinese

88.94

88.88

87.91

81.98

08.53

21

Czech

86.28

85.16

84.19

77.56

07.63

20

English

90.63

90.13

89.87

82.67

09.55

23

Greek

84.08

82.04

81.37

77.78

06.01

20

Hungarian

83.55

83.51

82.49

76.34

08.24

21

Italian

87.91

87.77

87.68

82.45

06.04

20

Turkish

86.22

85.77

85.77

80.33

06.34

20

Multilingual_average

86.55

85.71

85.62

71.43

23.86

23

pchemtb_closed

83.42

83.38

83.08

76.42

09.90

8

pchemtb_open

81.62

70.01

66.53

70.24

08.14

4

CHILDES_closed

61.37

58.89

58.67

57.89

02.79

6

CHILDES_open

62.49

60.27

45.61

56.12

09.17

3

LABEL ACCURACY

Task

1st

2nd

3rd

Avg

Std

Nteams

Arabic

86.55

85.73

85.64

81.79

04.89

20

Basque

83.62

82.52

81.34

76.07

05.97

20

Catalan

93.02

92.24

92.22

86.32

14.84

20

Chinese

87.06

86.22

86.03

80.16

07.47

21

Czech

86.56

85.41

85.29

79.66

06.95

20

English

93.76

93.12

92.40

87.69

06.56

23

Greek

85.47

84.30

84.16

81.26

04.62

20

Hungarian

90.85

90.62

89.98

85.89

05.41

21

Italian

90.05

89.87

89.62

85.75

04.54

20

Turkish

87.33

87.24

86.88

82.29

05.14

20

Multilingual_average

87.29

87.02

86.94

73.77

24.21

23

pchemtb_closed

88.28

87.44

87.40

81.74

07.97

8

pchemtb_open

87.02

76.62

72.87

77.14

06.88

4

Top3Scores (last edited 2007-05-02 07:07:38 by DenizYuret)