ImageCLEF
2005
Evaluation of image retrieval systems for historic photographic
and medical images
Interactive search
task
|
Background to cross-language image
retrieval |
The ImageCLEF interactive
search task provides user-centered evaluation of cross-language image retrieval
systems. Whereas the ad-hoc and medical tasks provide system-evaluation, the
interactive task aims to provide a framework in which groups can evaluate how
well their retrieval systems support user-interaction. The goal of ImageCLEF is
to establish how both visual features and texts associated with images can be
used for effective cross-language image retrieval.
In cross-language image
search, the object to be retrieved is an image. This is appealing as a CLIR
task because often (depending on the user and query) the object to be retrieved
(i.e. the image) can be assumed to be language-independent, i.e. there is no
need for further translation when presenting results to the user. This makes a
good introductory task to CLIR requiring only query translation to bridge the
language gap between the user's query (source) language, and the language used
to annotate the images (target language).
Image retrieval can be purely
visual in the case of query-by-example (QBE) which is entirely
language-independent, but this assumes the user wants to perform a visual
search (e.g. find me images which appear visually similar to the one provided).
However, users may also want to search for images starting with text-based
queries (e.g. Web image search) requiring that texts are associated with the
target image collection. For CLIR, the language of the texts used to annotate
the images should not affect retrieval, i.e. a user should be able to query the
images in their native language making the target language transparent.
Effective cross-language image retrieval will involve both text-based and
content-based IR (CBIR) methods in conjunction with translation.
The main areas of study for a cross-language image retrieval system
are:
| |
|
| (1)
|
How well a system supports user query formulation for
images with associated texts (e.g. captions or metadata) written in a language
different from the native language of the users. This is also an opportunity to
study how the images themselves could also be used as part of the query
formulation process. |
|
|
| (2)* |
How well a system supports query
re-formulation, e.g. the support of positive and negative feedback to improve
the user's search experience, and how this affects retrieval. This aims to
address issues such as how visual and textual features can be combined for
query reformulation/expansion. |
| |
|
| (3)* |
How well a system allows users to browse the image
collection. This might include support for summarising results (e.g. grouping
images by some pre-assigned categorization scheme or by visual feature such as
shape, colour or texture). Browsing becomes particularly important in a CLIR
system when query translation fails and returns irrelevant or no results.
|
| |
|
| (4) |
How well a system presents the retrieved results to the
user to enable the selection of relevant images. This might include how the
system presents the caption to the user (particularly if they are not familiar
with the language of the text associated with the images, or some of the
specific and colloquial language used in the captions) and investigate the
relationship between the image and caption for retrieval purposes. |
|
|
