IMAGECLEF 2003 submission -------------------------- Paul Clough, University of Sheffield (14th May 2003) Introduction ------------- Submission for the ImageCLEF task follows the standard TREC procedure and the relevant section from the general TREC guidelines have been reproduced almost verbatim below. Note in ImageCLEF, the document reference refers to both the image and caption. The format to use when submitting results is as follows, using a space as the delimiter between columns. The width of the columns in the format is not important, but it is important to include all columns and have at least one space between the columns. ... 25 1 stand03_118/stand03_20631 0 4238 xyzT10af5 25 1 stand03_668/stand03_20633 1 4223 xyzT10af5 25 1 stand03_268/stand03_12121 2 4207 xyzT10af5 25 1 stand03_68/stand03_12111 3 4194 xyzT10af5 25 1 stand03_1211/stand03_12121 4 4189 xyzT10af5 etc. where: (1) The first column is the topic number -- these will be numbered 1-50. (2) The second column is the query number within that topic and these allow for variation between the translations. This field is not used in TREC and is set to Q0, but this should be set to the "n" attribute within the title tag, e.g. "", in this case the second column will be 1. (3) The third column is the official document number of the retrieved document. This will take the form of: stand03_directory/stand03_image/caption, e.g. "stand03_668/stand03_20633". (4) The fourth column is the rank the document is retrieved. (5) The fifth column shows the score (integer or floating point) that generated the ranking. This score MUST be in descending (non-increasing) order and is important to include so that we can handle tied scores (for a given run) in a uniform fashion (the evaluation routines rank documents from these scores, not from your ranks). (6) The sixth column is called the "run tag" and should be a unique identifier for your group AND for the method used. That is, each run should have a different tag that identifies the group and the method that produced the run. Please use 12 or fewer letters and numbers, and NO punctuation, to facilitate labeling graphs and such with the tags. Submissions ----------- You can submit as many system runs for a given language as you like, however due to time and resource constraints we will only be able to provide relevance judgements on the run you state as your "best" entry. Evaluation of system runs will follow the standard TREC methodology. All submissions should be sent by email to Paul Clough (p.d.clough@sheffield.ac.uk). Deadlines ---------- The deadline for task 1 (automatic ad hoc retrieval) is 23rd May 2003; task 2 does not require formal submission. Results from the relevance judgements provided by Sheffield, and your report on either (or both) the automatic and interactive task will be published in the CLEF 2003 proceedings. The deadline for all paper submissions to CLEF is 20th July.