Login

Forgot password? Request a new one here!

Page navigation

Challenges »
BRATS2013

How to join BRATS 2013

Register below, select BRATS2013 as the research unit

How to join BRATS 2013 if you are already registered (e.g. in BRATS2012): Navigate to MySMIR, scroll to "Group Membership" apply for a new Membership by selecting BRATS2013
More information can be found at NCI-MICCAI 2013 Grand Challenges in Image Segmentation

Acknowledgements

Brain tumor image data used in this article were obtained from the MICCAI 2013 Challenge on Multimodal Brain Tumor Segmentation. The challenge database contain fully anonymized images from the Cancer Imaging Atlas Archive and the BRATS 2012 challenge.

Citation

The outcome of the BRATS2012 and BRATS2013 challenges has been summarized in the following publication. Authors using the BRATS dataset are kindly requested to cite this work:

  • Menze et al., The Multimodal Brain TumorImage Segmentation Benchmark (BRATS), IEEE Trans. Med. Imaging, 2015.Get the citation as BibTex
  • Kistler et. al, The virtual skeleton database: an open access repository for biomedical research and collaboration. JMIR, 2013. (BibTex)

Static Results 2012 & 2013

BRATS 2012 & 2013: Contributed algorithmic segmentations of the test data: Static Results and Testdata Download

Registration

You have to use your institutional email address for the registration.
Please note that while you can use non-institutional emails (e.g. Gmail, Hotmail, Yahoo, etc.) your request might be refused depending on the license model of the dataset (e.g. non-commercial use), hence, we recommend using an educational or non-profit account.
Register

Download

Please register to receive an email with your login link and activate your account.

BRATS 2013 Challenge data

10 patient datasets for download:

You need to log in to download the challenge data!

BRATS 2013 Leaderboard data

25 patient datasets for download:

You need to log in to download the leaderboard data!

BRATS 2013 Training data

30 patient datasets and 50 synthetic datasets including ground truths are available for download:

You need to log in to download the training data!

BRATS 2013 Individual Observer Ground-truth data

You need to log in to download the observer ground truth data!

Evaluation

Ground Truth images

All images are stored as signed 16-bit integers, but only positive values are used. The manual segmentations (Truth) of the patient images have the following four different labels:

    1 for necrosis
    2 for edema
    3 for non-enhancing tumor
    4 for enhancing tumor
    0 for everything else

IMPORTANT

here are 3 requirements for the successfull upload and validation of your segmentation:

  1. Use the MHA filetype to store your segmentations (not mhd) [use short or ushort if you experience any upload problems]
  2. Keep the same labels as the provided truth.mha (see above)
  3. Name your segmentations according to this template: VSD.your_description.###.mha

    replace the ### with the ID of the corresponding Flair MR images. This allows the system to relate your segmentation to the correct training truth. Download an example list for the training data and leaderboard data and challenge data.

Example 1
    Filename of the High Grade Flair 0001 MR: VSD.Brain.XX.O.MR_Flair.687.mha
     The VSD-Id would be 687
    Filename of your segmentation: VSD.Seg_HG_001.687.mha
Example 2
    Filename of the Low Grade 011 Flair MR: VSD.Brain.XX.O.MR_Flair.835.mha
     The VSD-Id would be 835
    Filename of your segmentation: VSD.Seg_LG_011.835.mha

Evaluation details

The evaluation is done for 3 different tumor sub-compartements:

  • Region 1: complete tumor (labels 1+2+3+4 for patient data, labesl 1+2 for synthetic data)
  • Region 2: Tumor core (labels 1+3+4 for patient data, label 2 for synthetic data)
  • Region 3: Enhancing tumor (label 4 for patient data, n.a. for synthetic data)

Evaluation Results: Challenge

Patient

Position Dice Positive Predictive Value Sensitivity Complete tumor Rank Tumor core Rank Enhancing tumor Rank
  complete core enhancing complete core enhancing complete core enhancing  
1 0.89 (1) 0.82 (4) 0.76 (3) 0.83 (37) 0.83 (20) 0.79 (10) 0.96 (1) 0.87 (3) 0.76 (24) 13.00 9.00 12.33
2 0.88 (5) 0.83 (2) 0.77 (2) 0.88 (14) 0.87 (7) 0.74 (26) 0.89 (33) 0.83 (10) 0.81 (11) 17.33 6.33 13.00
3 0.87 (12) 0.84 (1) 0.77 (1) 0.88 (16) 0.87 (6) 0.76 (18) 0.88 (38) 0.82 (12) 0.80 (13) 22.00 6.33 10.67
4 0.88 (4) 0.81 (10) 0.76 (5) 0.89 (9) 0.81 (28) 0.71 (38) 0.88 (36) 0.83 (9) 0.83 (5) 16.33 15.67 16.00
5 0.89 (2) 0.80 (12) 0.74 (13) 0.86 (19) 0.86 (13) 0.78 (13) 0.92 (18) 0.76 (33) 0.73 (36) 13.00 19.33 20.67
6 0.88 (9) 0.81 (8) 0.75 (8) 0.84 (33) 0.79 (36) 0.72 (34) 0.92 (16) 0.88 (1) 0.80 (14) 19.33 15.00 18.67
7 0.87 (15) 0.82 (6) 0.76 (4) 0.82 (39) 0.81 (27) 0.67 (50) 0.93 (13) 0.86 (5) 0.90 (1) 22.33 12.67 18.33
8 0.87 (17) 0.82 (5) 0.75 (7) 0.84 (31) 0.85 (17) 0.70 (43) 0.91 (21) 0.82 (14) 0.82 (8) 23.00 12.00 19.33
9 0.87 (11) 0.80 (11) 0.76 (6) 0.91 (5) 0.88 (4) 0.77 (16) 0.83 (54) 0.77 (31) 0.76 (26) 23.33 15.33 16.00
10 0.88 (3) 0.80 (13) 0.73 (26) 0.88 (11) 0.87 (8) 0.80 (5) 0.89 (30) 0.77 (30) 0.70 (44) 14.67 17.00 25.00
11 0.88 (7) 0.83 (3) 0.72 (28) 0.92 (4) 0.90 (2) 0.74 (27) 0.84 (50) 0.78 (25) 0.72 (40) 20.33 10.00 31.67
12 0.87 (10) 0.78 (15) 0.71 (33) 0.82 (40) 0.80 (31) 0.83 (1) 0.95 (3) 0.80 (19) 0.67 (52) 17.67 21.67 28.67
13 0.86 (24) 0.78 (19) 0.74 (10) 0.81 (43) 0.77 (40) 0.79 (11) 0.93 (11) 0.82 (15) 0.73 (32) 26.00 24.67 17.67
14 0.87 (20) 0.78 (18) 0.74 (11) 0.85 (25) 0.74 (51) 0.69 (45) 0.89 (28) 0.88 (2) 0.83 (6) 24.33 23.67 20.67
15 0.88 (8) 0.77 (24) 0.72 (27) 0.83 (34) 0.77 (42) 0.81 (3) 0.93 (6) 0.82 (13) 0.69 (49) 16.00 26.33 26.33
16 0.88 (6) 0.75 (34) 0.74 (12) 0.88 (12) 0.77 (41) 0.78 (12) 0.88 (35) 0.78 (27) 0.72 (38) 17.67 34.00 20.67
17 0.85 (30) 0.78 (14) 0.73 (18) 0.80 (47) 0.86 (10) 0.76 (20) 0.93 (10) 0.74 (41) 0.74 (30) 29.00 21.67 22.67
18 0.87 (16) 0.82 (7) 0.70 (40) 0.90 (6) 0.82 (23) 0.73 (30) 0.85 (48) 0.87 (4) 0.69 (48) 23.33 11.33 39.33
19 0.85 (31) 0.78 (16) 0.75 (9) 0.88 (15) 0.86 (12) 0.72 (35) 0.84 (51) 0.73 (43) 0.80 (12) 32.33 23.67 18.67
20 0.87 (14) 0.76 (30) 0.73 (16) 0.85 (23) 0.76 (43) 0.71 (39) 0.89 (31) 0.81 (16) 0.77 (18) 22.67 29.67 24.33
21 0.87 (13) 0.78 (17) 0.70 (36) 0.89 (7) 0.83 (22) 0.75 (23) 0.86 (45) 0.78 (24) 0.70 (46) 21.67 21.00 35.00
22 0.87 (18) 0.77 (21) 0.73 (24) 0.84 (28) 0.79 (35) 0.73 (28) 0.90 (26) 0.78 (26) 0.75 (27) 24.00 27.33 26.33
23 0.85 (29) 0.76 (27) 0.73 (17) 0.80 (46) 0.85 (14) 0.77 (14) 0.93 (9) 0.72 (47) 0.73 (34) 28.00 29.33 21.67
24 0.85 (34) 0.77 (23) 0.73 (21) 0.79 (50) 0.83 (21) 0.77 (15) 0.93 (5) 0.74 (40) 0.73 (33) 29.67 28.00 23.00
25 0.85 (40) 0.77 (26) 0.73 (19) 0.79 (51) 0.85 (15) 0.77 (17) 0.93 (7) 0.72 (45) 0.74 (31) 32.67 28.67 22.33
26 0.87 (19) 0.77 (25) 0.72 (29) 0.84 (27) 0.80 (33) 0.71 (37) 0.89 (29) 0.76 (34) 0.77 (21) 25.00 30.67 29.00
27 0.86 (21) 0.75 (35) 0.73 (25) 0.82 (41) 0.75 (47) 0.79 (7) 0.92 (14) 0.79 (23) 0.70 (43) 25.33 35.00 25.00
28 0.86 (28) 0.76 (28) 0.73 (20) 0.81 (42) 0.84 (18) 0.73 (31) 0.91 (20) 0.72 (48) 0.77 (22) 30.00 31.33 24.33
29 0.86 (22) 0.73 (44) 0.74 (14) 0.83 (36) 0.70 (57) 0.76 (21) 0.91 (19) 0.80 (17) 0.75 (29) 25.67 39.33 21.33
30 0.85 (33) 0.78 (20) 0.73 (23) 0.95 (1) 0.81 (25) 0.70 (42) 0.78 (61) 0.76 (36) 0.77 (20) 31.67 27.00 28.33
31 0.86 (26) 0.74 (40) 0.73 (15) 0.83 (35) 0.74 (49) 0.68 (49) 0.90 (24) 0.78 (28) 0.81 (9) 28.33 39.00 24.33
32 0.85 (41) 0.81 (9) 0.70 (38) 0.84 (29) 0.86 (11) 0.70 (40) 0.86 (44) 0.78 (29) 0.70 (42) 38.00 16.33 40.00
33 0.85 (37) 0.73 (43) 0.73 (22) 0.80 (48) 0.67 (61) 0.66 (52) 0.93 (12) 0.85 (6) 0.84 (4) 32.33 36.67 26.00
34 0.85 (32) 0.76 (29) 0.72 (32) 0.84 (30) 0.81 (26) 0.75 (24) 0.87 (40) 0.76 (35) 0.71 (41) 34.00 30.00 32.33
35 0.83 (47) 0.75 (33) 0.72 (31) 0.76 (58) 0.72 (55) 0.65 (56) 0.92 (17) 0.85 (7) 0.85 (3) 40.67 31.67 30.00
36 0.85 (35) 0.75 (37) 0.72 (30) 0.81 (44) 0.74 (50) 0.69 (46) 0.90 (25) 0.79 (21) 0.76 (25) 34.67 36.00 33.67
37 0.86 (27) 0.73 (45) 0.71 (34) 0.81 (45) 0.75 (45) 0.75 (22) 0.92 (15) 0.73 (44) 0.70 (45) 29.00 44.67 33.67
38 0.84 (44) 0.75 (36) 0.70 (35) 0.85 (24) 0.81 (29) 0.67 (51) 0.83 (53) 0.74 (39) 0.78 (16) 40.33 34.67 34.00
39 0.83 (48) 0.76 (31) 0.70 (37) 0.77 (54) 0.80 (30) 0.73 (32) 0.90 (22) 0.73 (42) 0.73 (35) 41.33 34.33 34.67
40 0.86 (23) 0.71 (47) 0.62 (60) 0.88 (17) 0.84 (19) 0.80 (6) 0.85 (46) 0.66 (57) 0.56 (61) 28.67 41.00 42.33
41 0.78 (56) 0.75 (32) 0.68 (46) 0.67 (64) 0.73 (53) 0.59 (63) 0.95 (2) 0.80 (18) 0.82 (7) 40.67 34.33 38.67
42 0.83 (46) 0.74 (42) 0.67 (49) 0.89 (8) 0.89 (3) 0.76 (19) 0.79 (59) 0.65 (58) 0.63 (59) 37.67 34.33 42.33
43 0.85 (38) 0.74 (41) 0.69 (44) 0.87 (18) 0.74 (52) 0.63 (58) 0.84 (52) 0.75 (38) 0.77 (19) 36.00 43.67 40.33
44 0.84 (45) 0.72 (46) 0.70 (39) 0.77 (55) 0.75 (46) 0.68 (48) 0.93 (8) 0.71 (51) 0.77 (23) 36.00 47.67 36.67
45 0.81 (53) 0.75 (38) 0.68 (45) 0.79 (49) 0.86 (9) 0.79 (8) 0.85 (49) 0.69 (54) 0.65 (57) 50.33 33.67 36.67
46 0.86 (25) 0.69 (53) 0.69 (41) 0.88 (13) 0.72 (56) 0.71 (36) 0.85 (47) 0.72 (50) 0.69 (50) 28.33 53.00 42.33
47 0.82 (50) 0.71 (49) 0.69 (43) 0.78 (52) 0.68 (60) 0.62 (59) 0.89 (34) 0.79 (20) 0.81 (10) 45.33 43.00 37.33
48 0.81 (52) 0.68 (56) 0.68 (47) 0.86 (20) 0.87 (5) 0.74 (25) 0.79 (58) 0.58 (64) 0.66 (53) 43.33 41.67 41.67
49 0.78 (55) 0.71 (48) 0.64 (58) 0.92 (3) 0.80 (34) 0.79 (9) 0.72 (66) 0.68 (55) 0.58 (60) 41.33 45.67 42.33
50 0.83 (49) 0.70 (51) 0.61 (61) 0.86 (21) 0.80 (32) 0.82 (2) 0.81 (57) 0.66 (56) 0.56 (62) 42.33 46.33 41.67
51 0.84 (43) 0.68 (54) 0.65 (56) 0.92 (2) 0.85 (16) 0.70 (44) 0.78 (60) 0.61 (61) 0.66 (55) 35.00 43.67 51.67
52 0.84 (42) 0.68 (57) 0.65 (54) 0.88 (10) 0.63 (63) 0.68 (47) 0.81 (56) 0.82 (11) 0.66 (54) 36.00 43.67 51.67
53 0.77 (57) 0.74 (39) 0.65 (52) 0.66 (66) 0.79 (37) 0.62 (60) 0.94 (4) 0.72 (46) 0.72 (37) 42.33 40.67 49.67
54 0.74 (62) 0.54 (65) 0.56 (63) 0.85 (22) 0.92 (1) 0.81 (4) 0.73 (63) 0.46 (67) 0.51 (65) 49.00 44.33 44.00
55 0.74 (63) 0.68 (55) 0.58 (62) 0.68 (63) 0.70 (59) 0.49 (66) 0.90 (27) 0.79 (22) 0.86 (2) 51.00 45.33 43.33
56 0.85 (36) 0.58 (63) 0.65 (53) 0.83 (38) 0.58 (64) 0.58 (64) 0.88 (37) 0.70 (52) 0.79 (15) 37.00 59.67 44.00
57 0.75 (61) 0.70 (50) 0.67 (50) 0.66 (67) 0.67 (62) 0.61 (61) 0.90 (23) 0.76 (32) 0.78 (17) 50.33 48.00 42.67
58 0.82 (51) 0.77 (22) 0.00 (69) 0.76 (57) 0.74 (48) 0.00 (69) 0.89 (32) 0.84 (8) 0.00 (69) 46.67 26.00 69.00
59 0.85 (39) 0.68 (58) 0.65 (55) 0.84 (32) 0.76 (44) 0.66 (53) 0.87 (41) 0.63 (60) 0.69 (51) 37.33 54.00 53.00
60 0.76 (59) 0.69 (52) 0.69 (42) 0.69 (61) 0.70 (58) 0.65 (55) 0.87 (42) 0.76 (37) 0.75 (28) 54.00 49.00 41.67
61 0.77 (58) 0.64 (60) 0.64 (57) 0.85 (26) 0.73 (54) 0.73 (29) 0.72 (64) 0.59 (63) 0.64 (58) 49.33 59.00 48.00
62 0.79 (54) 0.62 (61) 0.48 (66) 0.78 (53) 0.82 (24) 0.72 (33) 0.82 (55) 0.55 (65) 0.42 (66) 54.00 50.00 55.00
63 0.72 (64) 0.66 (59) 0.67 (48) 0.77 (56) 0.77 (38) 0.70 (41) 0.72 (65) 0.60 (62) 0.70 (47) 61.67 53.00 45.33
64 0.70 (66) 0.57 (64) 0.65 (51) 0.68 (62) 0.53 (65) 0.65 (57) 0.74 (62) 0.65 (59) 0.72 (39) 63.33 62.67 49.00
65 0.69 (67) 0.59 (62) 0.62 (59) 0.71 (60) 0.77 (39) 0.65 (54) 0.69 (67) 0.49 (66) 0.65 (56) 64.67 55.67 56.33
66 0.76 (60) 0.52 (66) 0.54 (64) 0.71 (59) 0.42 (66) 0.61 (62) 0.87 (43) 0.72 (49) 0.53 (64) 54.00 60.33 63.33
67 0.71 (65) 0.46 (67) 0.52 (65) 0.66 (65) 0.38 (67) 0.58 (65) 0.87 (39) 0.70 (53) 0.55 (63) 56.33 62.33 64.33
68 0.09 (68) 0.09 (68) 0.08 (67) 0.09 (68) 0.09 (68) 0.08 (67) 0.08 (69) 0.09 (68) 0.09 (67) 68.33 68.00 67.00
69 0.06 (69) 0.00 (69) 0.00 (68) 0.06 (69) 0.01 (69) 0.00 (68) 0.13 (68) 0.00 (69) 0.00 (68) 68.67 69.00 68.00

Evaluation Results: Leaderboard

Patient

Position Dice Positive Predictive Value Sensitivity Complete tumor Rank Tumor core Rank Enhancing tumor Rank
  complete core enhancing complete core enhancing complete core enhancing  
1 0.84 (6) 0.72 (5) 0.62 (1) 0.85 (7) 0.82 (4) 0.60 (7) 0.86 (18) 0.76 (10) 0.68 (8) 10.33 6.33 5.33
2 0.84 (5) 0.71 (8) 0.60 (6) 0.83 (9) 0.72 (15) 0.60 (10) 0.90 (7) 0.80 (5) 0.71 (2) 7.00 9.33 6.00
3 0.86 (2) 0.79 (1) 0.59 (11) 0.88 (2) 0.84 (3) 0.60 (8) 0.86 (20) 0.81 (4) 0.63 (22) 8.00 2.67 13.67
4 0.83 (9) 0.73 (3) 0.62 (2) 0.91 (1) 0.76 (8) 0.64 (2) 0.78 (38) 0.77 (7) 0.68 (9) 16.00 6.00 4.33
5 0.87 (1) 0.70 (10) 0.61 (3) 0.86 (5) 0.62 (31) 0.60 (9) 0.91 (5) 0.93 (1) 0.65 (16) 3.67 14.00 9.33
6 0.86 (3) 0.70 (9) 0.60 (7) 0.82 (11) 0.77 (7) 0.57 (12) 0.91 (4) 0.68 (23) 0.68 (10) 6.00 13.00 9.67
7 0.78 (20) 0.70 (11) 0.61 (4) 0.72 (24) 0.71 (16) 0.57 (13) 0.90 (6) 0.76 (8) 0.71 (3) 16.67 11.67 6.67
8 0.84 (7) 0.68 (12) 0.59 (9) 0.86 (4) 0.86 (1) 0.61 (6) 0.84 (24) 0.65 (34) 0.68 (12) 11.67 15.67 9.00
9 0.84 (4) 0.79 (2) 0.50 (34) 0.79 (16) 0.85 (2) 0.42 (42) 0.94 (2) 0.79 (6) 0.70 (5) 7.33 3.33 27.00
10 0.83 (10) 0.73 (4) 0.55 (20) 0.77 (21) 0.67 (23) 0.46 (37) 0.94 (1) 0.89 (2) 0.78 (1) 10.67 9.67 19.33
11 0.82 (12) 0.66 (17) 0.60 (5) 0.82 (10) 0.77 (6) 0.61 (5) 0.87 (16) 0.66 (31) 0.63 (23) 12.67 18.00 11.00
12 0.82 (13) 0.67 (15) 0.59 (10) 0.81 (12) 0.68 (18) 0.56 (17) 0.87 (13) 0.70 (17) 0.65 (17) 12.67 16.67 14.67
13 0.83 (11) 0.71 (7) 0.54 (24) 0.85 (6) 0.73 (13) 0.59 (11) 0.84 (25) 0.82 (3) 0.58 (32) 14.00 7.67 22.33
14 0.83 (8) 0.67 (14) 0.57 (15) 0.88 (3) 0.75 (9) 0.65 (1) 0.83 (29) 0.69 (20) 0.57 (35) 13.33 14.33 17.00
15 0.79 (17) 0.67 (16) 0.60 (8) 0.80 (13) 0.74 (11) 0.56 (16) 0.81 (33) 0.68 (24) 0.70 (4) 21.00 17.00 9.33
16 0.78 (21) 0.64 (20) 0.57 (16) 0.71 (26) 0.66 (25) 0.63 (4) 0.93 (3) 0.73 (12) 0.59 (31) 16.67 19.00 17.00
17 0.79 (16) 0.67 (13) 0.54 (22) 0.77 (20) 0.75 (10) 0.55 (20) 0.86 (19) 0.70 (16) 0.62 (25) 18.33 13.00 22.33
18 0.79 (18) 0.71 (6) 0.50 (33) 0.73 (23) 0.77 (5) 0.50 (29) 0.88 (12) 0.76 (9) 0.60 (28) 17.67 6.67 30.00
19 0.79 (15) 0.65 (19) 0.53 (30) 0.83 (8) 0.70 (17) 0.51 (27) 0.81 (34) 0.73 (11) 0.66 (14) 19.00 15.67 23.67
20 0.75 (26) 0.62 (24) 0.57 (13) 0.69 (30) 0.68 (20) 0.53 (22) 0.89 (8) 0.64 (36) 0.69 (7) 21.33 26.67 14.00
21 0.76 (22) 0.61 (27) 0.55 (19) 0.74 (22) 0.68 (19) 0.64 (3) 0.86 (17) 0.67 (28) 0.56 (36) 20.33 24.67 19.33
22 0.76 (24) 0.64 (21) 0.56 (17) 0.69 (29) 0.67 (22) 0.55 (18) 0.89 (9) 0.67 (27) 0.62 (27) 20.67 23.33 20.67
23 0.78 (19) 0.66 (18) 0.54 (25) 0.77 (19) 0.74 (12) 0.52 (24) 0.84 (26) 0.68 (25) 0.62 (26) 21.33 18.33 25.00
24 0.73 (28) 0.64 (22) 0.57 (12) 0.71 (27) 0.66 (24) 0.53 (23) 0.82 (30) 0.68 (26) 0.68 (11) 28.33 24.00 15.33
25 0.68 (37) 0.63 (23) 0.57 (14) 0.61 (42) 0.65 (27) 0.56 (15) 0.87 (14) 0.69 (21) 0.64 (21) 31.00 23.67 16.67
26 0.71 (33) 0.61 (26) 0.53 (31) 0.63 (39) 0.58 (34) 0.50 (30) 0.89 (10) 0.72 (13) 0.64 (19) 27.33 24.33 26.67
27 0.72 (31) 0.60 (29) 0.53 (32) 0.65 (34) 0.62 (29) 0.48 (33) 0.88 (11) 0.69 (19) 0.64 (18) 25.33 25.67 27.67
28 0.76 (23) 0.60 (30) 0.53 (28) 0.78 (17) 0.62 (30) 0.57 (14) 0.79 (36) 0.67 (29) 0.55 (37) 25.33 29.67 26.33
29 0.72 (30) 0.58 (33) 0.55 (21) 0.69 (31) 0.63 (28) 0.52 (25) 0.85 (21) 0.66 (32) 0.63 (24) 27.33 31.00 23.33
30 0.73 (29) 0.62 (25) 0.54 (27) 0.72 (25) 0.65 (26) 0.49 (31) 0.80 (35) 0.65 (33) 0.66 (15) 29.67 28.00 24.33
31 0.69 (35) 0.59 (31) 0.56 (18) 0.66 (33) 0.67 (21) 0.53 (21) 0.76 (41) 0.56 (40) 0.66 (13) 36.33 30.67 17.33
32 0.69 (36) 0.58 (32) 0.54 (23) 0.64 (36) 0.54 (40) 0.51 (28) 0.82 (31) 0.72 (15) 0.64 (20) 34.33 29.00 23.67
33 0.80 (14) 0.54 (38) 0.46 (39) 0.78 (18) 0.49 (43) 0.47 (34) 0.85 (22) 0.70 (18) 0.53 (39) 18.00 33.00 37.33
34 0.70 (34) 0.57 (34) 0.54 (26) 0.65 (35) 0.55 (39) 0.52 (26) 0.87 (15) 0.67 (30) 0.60 (29) 28.00 34.33 27.00
35 0.75 (25) 0.61 (28) 0.46 (40) 0.79 (15) 0.60 (32) 0.43 (41) 0.78 (37) 0.72 (14) 0.52 (40) 25.67 24.67 40.33
36 0.72 (32) 0.55 (37) 0.49 (35) 0.67 (32) 0.53 (41) 0.46 (36) 0.83 (28) 0.68 (22) 0.60 (30) 30.67 33.33 33.67
37 0.65 (41) 0.49 (42) 0.43 (42) 0.80 (14) 0.73 (14) 0.55 (19) 0.59 (45) 0.45 (43) 0.40 (46) 33.33 33.00 35.67
38 0.66 (40) 0.54 (39) 0.53 (29) 0.60 (43) 0.57 (35) 0.46 (35) 0.76 (40) 0.58 (39) 0.69 (6) 41.00 37.67 23.33
39 0.74 (27) 0.55 (36) 0.46 (38) 0.71 (28) 0.55 (38) 0.41 (43) 0.82 (32) 0.64 (35) 0.58 (33) 29.00 36.33 38.00
40 0.67 (38) 0.56 (35) 0.47 (37) 0.62 (40) 0.59 (33) 0.49 (32) 0.84 (23) 0.60 (37) 0.51 (41) 33.67 35.00 36.67
41 0.67 (39) 0.50 (41) 0.43 (41) 0.63 (38) 0.51 (42) 0.41 (44) 0.83 (27) 0.60 (38) 0.57 (34) 34.67 40.33 39.67
42 0.64 (42) 0.52 (40) 0.48 (36) 0.64 (37) 0.55 (37) 0.45 (39) 0.70 (42) 0.56 (41) 0.54 (38) 40.33 39.33 37.67
43 0.63 (43) 0.42 (43) 0.42 (43) 0.61 (41) 0.39 (46) 0.45 (38) 0.68 (43) 0.48 (42) 0.43 (44) 42.33 43.67 41.67
44 0.55 (45) 0.38 (46) 0.41 (44) 0.59 (44) 0.57 (36) 0.44 (40) 0.57 (46) 0.33 (47) 0.42 (45) 45.00 43.00 43.00
45 0.51 (46) 0.40 (44) 0.40 (45) 0.47 (46) 0.45 (44) 0.40 (45) 0.76 (39) 0.43 (44) 0.50 (42) 43.67 44.00 44.00
46 0.56 (44) 0.40 (45) 0.39 (46) 0.54 (45) 0.43 (45) 0.36 (46) 0.61 (44) 0.41 (45) 0.45 (43) 44.33 45.00 45.00
47 0.34 (48) 0.34 (47) 0.31 (47) 0.30 (50) 0.32 (50) 0.27 (48) 0.39 (48) 0.37 (46) 0.38 (47) 48.67 47.67 47.33
48 0.38 (47) 0.30 (50) 0.26 (50) 0.41 (47) 0.34 (48) 0.26 (50) 0.40 (47) 0.32 (48) 0.32 (48) 47.00 48.67 49.33
49 0.33 (50) 0.32 (48) 0.29 (48) 0.35 (49) 0.35 (47) 0.34 (47) 0.32 (50) 0.30 (49) 0.26 (50) 49.67 48.00 48.33
50 0.33 (49) 0.30 (49) 0.29 (49) 0.30 (51) 0.34 (49) 0.27 (49) 0.38 (49) 0.28 (50) 0.31 (49) 49.67 49.33 49.00
51 0.26 (51) 0.16 (51) 0.00 (51) 0.38 (48) 0.18 (51) 0.00 (51) 0.23 (51) 0.21 (51) 0.00 (51) 50.00 51.00 51.00
52 0.07 (52) 0.04 (52) 0.00 (51) 0.07 (52) 0.05 (52) 0.00 (51) 0.10 (52) 0.07 (52) 0.00 (51) 52.00 52.00 51.00

Evaluation Results: Training

Patient

Position Dice Positive Predictive Value Sensitivity Kappa Complete tumor Rank Tumor core Rank Enhancing tumor Rank
  complete core enhancing complete core enhancing complete core enhancing  
1 0.92 (1) 0.91 (2) 0.68 (1) 0.97 (1) 0.95 (1) 0.70 (1) 0.89 (7) 0.88 (4) 0.66 (3) 1.00 (1) 2.50 2.25 7.50
2 0.92 (2) 0.93 (1) 0.63 (2) 0.86 (13) 0.92 (2) 0.63 (7) 0.98 (1) 0.94 (1) 0.64 (4) 1.00 (3) 4.75 1.25 9.75
3 0.87 (7) 0.87 (4) 0.61 (3) 0.78 (22) 0.83 (8) 0.57 (14) 0.98 (2) 0.93 (2) 0.66 (2) 1.00 (8) 9.75 4.25 5.00
4 0.87 (5) 0.73 (8) 0.58 (4) 0.93 (3) 0.86 (7) 0.62 (9) 0.84 (12) 0.69 (12) 0.61 (6) 1.00 (5) 6.25 9.25 5.25
5 0.88 (4) 0.76 (6) 0.55 (9) 0.88 (9) 0.80 (11) 0.65 (5) 0.89 (6) 0.79 (7) 0.53 (10) 1.00 (4) 5.75 7.50 10.00
6 0.91 (3) 0.88 (3) 0.45 (25) 0.91 (6) 0.88 (4) 0.37 (29) 0.92 (4) 0.89 (3) 0.68 (1) 1.00 (2) 3.75 3.50 16.75
7 0.84 (12) 0.71 (9) 0.56 (7) 0.92 (4) 0.78 (13) 0.64 (6) 0.78 (24) 0.69 (11) 0.51 (11) 1.00 (7) 11.75 9.50 6.75
8 0.86 (8) 0.75 (7) 0.55 (10) 0.88 (10) 0.86 (6) 0.65 (3) 0.86 (10) 0.72 (9) 0.50 (15) 1.00 (10) 9.50 7.75 11.75
9 0.86 (9) 0.70 (10) 0.54 (11) 0.94 (2) 0.88 (3) 0.65 (4) 0.81 (20) 0.66 (19) 0.54 (8) 1.00 (6) 9.25 10.75 11.50
10 0.87 (6) 0.79 (5) 0.53 (12) 0.92 (5) 0.82 (10) 0.56 (15) 0.83 (14) 0.80 (6) 0.54 (9) 1.00 (9) 8.50 7.00 16.25
11 0.83 (14) 0.67 (13) 0.55 (8) 0.90 (8) 0.77 (14) 0.66 (2) 0.80 (22) 0.69 (14) 0.51 (12) 1.00 (12) 14.00 13.50 10.50
12 0.79 (20) 0.68 (12) 0.57 (6) 0.81 (20) 0.75 (16) 0.54 (18) 0.79 (23) 0.67 (16) 0.63 (5) 1.00 (15) 19.50 13.00 8.50
13 0.79 (18) 0.62 (20) 0.57 (5) 0.82 (19) 0.80 (12) 0.60 (10) 0.80 (21) 0.57 (23) 0.57 (7) 1.00 (17) 18.75 17.50 6.50
14 0.83 (15) 0.66 (14) 0.52 (14) 0.85 (15) 0.74 (17) 0.59 (13) 0.83 (15) 0.66 (17) 0.50 (16) 1.00 (14) 14.75 15.50 15.00
15 0.85 (10) 0.58 (22) 0.48 (20) 0.88 (11) 0.76 (15) 0.60 (12) 0.85 (11) 0.53 (27) 0.46 (22) 1.00 (11) 10.75 20.50 15.00
16 0.83 (13) 0.54 (24) 0.48 (18) 0.84 (17) 0.47 (27) 0.51 (23) 0.86 (9) 0.77 (8) 0.51 (13) 1.00 (16) 13.75 21.00 15.50
17 0.81 (16) 0.69 (11) 0.49 (16) 0.78 (24) 0.63 (22) 0.54 (17) 0.89 (5) 0.85 (5) 0.46 (21) 0.99 (21) 16.50 15.00 18.75
18 0.78 (21) 0.66 (16) 0.46 (23) 0.66 (27) 0.87 (5) 0.62 (8) 0.97 (3) 0.56 (24) 0.40 (28) 0.99 (23) 18.50 14.25 18.50
19 0.79 (19) 0.64 (18) 0.49 (15) 0.79 (21) 0.74 (18) 0.54 (19) 0.82 (16) 0.61 (20) 0.49 (18) 0.99 (20) 19.00 18.00 16.50
20 0.79 (17) 0.66 (15) 0.48 (17) 0.90 (7) 0.73 (19) 0.52 (22) 0.73 (27) 0.66 (18) 0.49 (19) 1.00 (19) 17.50 17.25 19.00
21 0.77 (22) 0.63 (19) 0.47 (21) 0.84 (16) 0.67 (21) 0.56 (16) 0.75 (26) 0.69 (10) 0.45 (24) 1.00 (18) 20.50 17.75 17.00
22 0.84 (11) 0.65 (17) 0.44 (26) 0.86 (12) 0.70 (20) 0.51 (24) 0.84 (13) 0.69 (15) 0.42 (27) 1.00 (13) 12.25 18.00 26.25
23 0.68 (27) 0.62 (21) 0.52 (13) 0.78 (23) 0.83 (9) 0.60 (11) 0.62 (29) 0.53 (26) 0.50 (14) 0.99 (22) 25.25 18.75 16.25
24 0.73 (23) 0.53 (25) 0.35 (29) 0.85 (14) 0.62 (23) 0.52 (21) 0.69 (28) 0.55 (25) 0.32 (33) 0.99 (24) 22.25 24.00 23.25
25 0.73 (24) 0.46 (27) 0.33 (30) 0.69 (26) 0.46 (29) 0.37 (30) 0.82 (18) 0.53 (28) 0.34 (31) 0.99 (25) 23.25 27.00 25.50
26 0.46 (34) 0.36 (31) 0.35 (28) 0.35 (34) 0.27 (33) 0.35 (31) 0.86 (8) 0.69 (13) 0.39 (29) 0.98 (29) 26.25 26.25 24.25
27 0.61 (28) 0.44 (29) 0.27 (33) 0.52 (32) 0.38 (31) 0.24 (32) 0.81 (19) 0.57 (22) 0.35 (30) 0.99 (27) 26.50 27.00 27.00
28 0.57 (30) 0.47 (26) 0.48 (19) 0.57 (29) 0.53 (25) 0.52 (20) 0.57 (31) 0.47 (30) 0.49 (20) 0.66 (31) 30.25 28.25 22.50
29 0.55 (31) 0.55 (23) 0.47 (22) 0.61 (28) 0.60 (24) 0.49 (25) 0.51 (33) 0.51 (29) 0.49 (17) 0.66 (32) 31.00 26.75 24.25
30 0.70 (25) 0.28 (33) 0.33 (31) 0.70 (25) 0.46 (28) 0.45 (27) 0.75 (25) 0.28 (33) 0.34 (32) 0.99 (26) 25.25 30.25 28.00
31 0.57 (29) 0.33 (32) 0.28 (32) 0.49 (33) 0.31 (32) 0.23 (33) 0.82 (17) 0.57 (21) 0.44 (25) 0.98 (28) 26.75 28.50 28.50
32 0.52 (32) 0.44 (28) 0.46 (24) 0.55 (31) 0.52 (26) 0.48 (26) 0.52 (32) 0.40 (31) 0.45 (23) 0.66 (33) 32.00 29.50 26.25
33 0.49 (33) 0.41 (30) 0.42 (27) 0.57 (30) 0.45 (30) 0.45 (28) 0.44 (34) 0.40 (32) 0.44 (26) 0.66 (34) 32.75 31.50 28.75
34 0.69 (26) 0.15 (34) 0.00 (42) 0.83 (18) 0.18 (34) 0.00 (42) 0.61 (30) 0.19 (34) 0.00 (42) 0.86 (30) 26.00 33.00 39.00
35 0.14 (35) 0.13 (35) 0.11 (34) 0.15 (35) 0.16 (35) 0.12 (34) 0.12 (35) 0.12 (35) 0.11 (34) 0.17 (35) 35.00 35.00 34.25
36 0.11 (36) 0.05 (36) 0.03 (35) 0.13 (36) 0.05 (36) 0.07 (35) 0.10 (36) 0.06 (37) 0.03 (35) 0.13 (36) 36.00 36.25 35.25
37 0.03 (38) 0.03 (38) 0.03 (36) 0.03 (39) 0.03 (37) 0.03 (36) 0.03 (38) 0.03 (39) 0.03 (37) 0.03 (38) 38.25 38.00 36.75
38 0.06 (37) 0.03 (37) 0.00 (42) 0.06 (37) 0.03 (40) 0.00 (42) 0.07 (37) 0.08 (36) 0.00 (42) 0.10 (37) 37.00 37.50 40.75
39 0.03 (41) 0.03 (39) 0.03 (37) 0.03 (40) 0.03 (38) 0.03 (38) 0.03 (43) 0.03 (41) 0.03 (36) 0.03 (42) 41.50 39.50 38.00
40 0.03 (40) 0.03 (40) 0.03 (38) 0.03 (41) 0.03 (39) 0.03 (39) 0.03 (41) 0.03 (40) 0.03 (38) 0.03 (41) 40.75 40.00 39.25
41 0.03 (39) 0.03 (41) 0.02 (39) 0.03 (38) 0.03 (41) 0.03 (37) 0.03 (42) 0.03 (42) 0.02 (41) 0.03 (40) 39.75 41.75 39.25
42 0.03 (43) 0.03 (42) 0.02 (40) 0.02 (43) 0.03 (42) 0.02 (40) 0.03 (40) 0.03 (43) 0.03 (40) 0.03 (43) 42.25 42.25 40.75
43 0.02 (44) 0.02 (43) 0.01 (41) 0.02 (44) 0.01 (44) 0.01 (41) 0.03 (39) 0.03 (38) 0.03 (39) 0.03 (44) 42.75 42.50 41.25
44 0.03 (42) 0.00 (44) 0.00 (42) 0.03 (42) 0.01 (43) 0.00 (42) 0.03 (44) 0.00 (44) 0.00 (42) 0.03 (39) 41.75 42.50 42.75
45 0.00 (45) 0.00 (45) 0.00 (42) 0.00 (45) 0.00 (45) 0.00 (42) 0.00 (45) 0.00 (45) 0.00 (42) 0.03 (45) 45.00 44.75 41.25
46 0.00 (45) 0.00 (45) 0.00 (42) 0.00 (45) 0.00 (45) 0.00 (42) 0.00 (45) 0.00 (45) 0.00 (42) 0.00 (46) 45.25 45.25 42.75

Simulated

Position Dice Positive Predictive Value Sensitivity Kappa Complete tumor Rank Tumor core Rank Enhancing tumor Rank
  complete core enhancing complete core enhancing complete core enhancing  
1 0.93 (1) 0.95 (1) 0.00 (1) 0.88 (2) 0.95 (1) 0.00 (1) 0.97 (3) 0.95 (1) 0.00 (1) 1.00 (2) 2.00 1.00 1.00
2 0.89 (3) 0.90 (2) 0.00 (1) 0.82 (5) 0.88 (2) 0.00 (1) 0.99 (1) 0.93 (2) 0.00 (1) 1.00 (3) 3.00 2.00 1.00
3 0.92 (2) 0.87 (3) 0.00 (1) 0.87 (3) 0.88 (3) 0.00 (1) 0.98 (2) 0.90 (3) 0.00 (1) 1.00 (1) 2.00 3.00 1.00
4 0.88 (4) 0.82 (4) 0.00 (1) 0.82 (4) 0.84 (4) 0.00 (1) 0.96 (4) 0.84 (5) 0.00 (1) 1.00 (4) 4.00 4.25 1.00
5 0.86 (5) 0.78 (5) 0.00 (1) 0.89 (1) 0.77 (5) 0.00 (1) 0.87 (6) 0.84 (4) 0.00 (1) 1.00 (5) 4.25 4.75 1.00
6 0.27 (6) 0.12 (6) 0.00 (1) 0.20 (6) 0.07 (6) 0.00 (1) 0.50 (7) 0.41 (7) 0.00 (1) 0.98 (6) 6.25 6.25 1.00
7 0.18 (7) 0.07 (7) 0.00 (1) 0.11 (7) 0.04 (7) 0.00 (1) 0.96 (5) 0.45 (6) 0.00 (1) 0.91 (7) 6.50 6.75 1.00
8 0.06 (8) 0.05 (8) 0.00 (1) 0.06 (8) 0.04 (8) 0.00 (1) 0.06 (8) 0.06 (8) 0.00 (1) 0.08 (8) 8.00 8.00 1.00