IMAGECLEF 2004 submission
--------------------------
Paul Clough, University of Sheffield (May 2004)


Introduction
-------------
Submission for the ImageCLEF task follows the standard TREC procedure and 
the relevant section from the general TREC guidelines have been reproduced 
almost verbatim below. Note in ImageCLEF, the document reference refers
to both the image and caption. 

The format to use when submitting results is as follows, using a space as 
the delimiter between columns. The width of the columns in the format is not 
important, but it is important to include all columns and have at least one 
space between the columns. 

e.g. for the bilingual ad hoc task:
   ...
   25 1 stand03_118/stand03_20631  0 4238 xyzT10af5
   25 1 stand03_668/stand03_20633  1 4223 xyzT10af5
   25 1 stand03_268/stand03_12121  2 4207 xyzT10af5
   25 1 stand03_68/stand03_12111  3 4194 xyzT10af5
   25 1 stand03_1211/stand03_12121  4 4189 xyzT10af5
      etc.

or for the medical task:
   ...
   1 1 f_11/10952 22 0.524393 GE_4d_16g_qe3
   1 1 f_9/8970 23 0.522450 GE_4d_16g_qe3
   1 1 f_10/10341 24 0.518187 GE_4d_16g_qe3
   1 1 f_10/10082 25 0.518032 GE_4d_16g_qe3
   1 1 f_12/12354 26 0.518029 GE_4d_16g_qe3

where:

(1) The first column is the topic number -- these will be numbered 1-25.

(2) The second column is the query number within that topic and these allow
    for variation between the translations. This field is not used in ImageCLEF
    2004 and should be set to 1.

(3) The third column is the official document number of the retrieved document.
    This will take the form of: directory/image, e.g. "stand03_668/stand03_20633" 
    (for ad hoc), or "f_11/10952" (for the medical task).
     
(4) The fourth column is the rank the document is retrieved (starting from 0).

(5) The fifth column shows the score (integer or floating point) that generated 
    the ranking. This score MUST be in descending (non-increasing) order and is 
    important to include so that we can handle tied scores (for a given run) in 
    a uniform fashion (the evaluation routines rank documents from these scores, 
    not from your ranks). 

(6) The sixth column is called the "run tag" and should be a unique identifier 
    for your group AND for the method used. That is, each run should have a 
    different tag that identifies the group and the method that produced the 
    run. Please use 12 or fewer letters and numbers, and NO punctuation, to 
    facilitate labeling graphs and such with the tags. 



Submissions
-----------

You can submit as many system runs for a given language as you like (ad hoc),
or system setting (medical task). We will publish your best result for a given
query setting (e.g. automatic vs. manual). Evaluation of system runs will follow 
the standard TREC methodology. 

All submissions should be sent by email to Paul Clough (p.d.clough@sheffield.ac.uk).


Deadlines
----------

The deadline for submission is 31st May 2004 for the ad hoc and medical tasks.

Results from the relevance judgements provided by Sheffield, and your report on 
either (or both) the automatic and interactive task will be published in the CLEF 
2004 proceedings. 

The deadline for all paper submissions to CLEF is 15th August 2004.