Ontology-based management of ancient "lettrines"

Tracking #: 2350-3563

Alain Bouju
Mickaël Coustaty

Responsible editor: 
Special Issue Cultural Heritage 2019

Submission type: 
Full Paper
A large amount of historical documents have been digitized over the years. Browsing into these Cultural Heritage data can be done using query by keywords or query by example systems. Going from one kind of query to another raises the problem of the semantic gap. Identify region inside image could be needed. In order to deal with this problem, this paper presents an ontology-based approach to the resolution of the semantic gap problem that uses a semantic web approach with historical images. To do this, historians' knowledge and knowledge from the document processing domain were modeled using dedicated ontologies. Then, links between the regions of interest from the computer vision algorithms on the one hand, and their meaning on the other hand, were created. These links will subsequently be used to help historians to search, query, analyze and enrich images dataset. Based on the three ontologies defined or reused and combined in this approach, we have defined rules to automatically annotate an image (to define the background for example), or a part of an image (to identify a letter, a body-part, ...).
Full PDF Version: 


Solicited Reviews:
Click to Expand/Collapse
Review #1
By Rafael Penaloza submitted on 14/Dec/2019
Review Comment:

This paper deals with a very important problem from information retrieval based
on images, which is known as the semantic gap. The semantic gap arises from the
different ways in which an image database can be queried. Given one or a few
images, one may ask for similar images (query by example), which is solved by
identifying and comparing features in an image. Alternatively, one may specify
a set of properties (keywords) of the images to be retrieved. Since features
and keywords are not interrelated, one cannot go from one of these queries to
the other.

This work uses three ontologies to give a semantic relationship between features
and keywords. Keywords are organized through a high-level ontology which can be
defined by the user to specify the meaning of these keywords. Features are
extracted automatically through previously existing approaches, and also organized
in a low-level ontology, taking many properties of the image into account (e.g.,
location, density, etc.). Finally, the connection between these two ontologies
is specified through a linking ontology. The overall result are three rather
simple TBoxes, which are then populated with large ABoxes.

The authors apply this general approach to create an ontology of lettrines from
medieval texts, to allow experts to query and retrieve important information from
existing data. It is important to note that the construction of the ontology
requires much manual work, not only in defining the TBox (which is small) but
also in populating the ABox. In fact, although the authors use automatic feature
extraction methods, they need to be manually verified. Still, the result is an
ontology which can be queries at both levels: from its features, or from its
keyword (or both at the same time), as was required by the original problem.

This is a pretty nice work. Although it does not give any technical innovations,
it shows how existing techniques can be used for practical problems, and also
points issues that need to be solved towards this goal. The application scenario
is very interesting and within the scope of this special issue. I have no
doubt that is can be accepted.

I have only a few (pedantic) remarks:

- "T-Box", "Tbox", etc. -> "TBox"
- the same for A-Box, A Box, etc
- Section 2.2: "Ontologies approach have" -> "Ontology approaches have"
- "[32], image content was modeled ..." -> I could not understand this sentence
- end of Section 4.4: if something is correct 17 out of 18 times, the error
rate is about 5%, not 2%

Review #2
Anonymous submitted on 12/Jan/2020
Review Comment:

This paper can be interpreted as a re-submission of the following journal paper published in 2015 in Applied Ontology:

Coustaty, M., Tsopze, N., Bouju, A., Bertet, K., and Louis, G.
(2015). Towards ontology-based retrieval of historical images.
Applied Ontology, 10(2):147–167.

Except Figure 1, all the 14 other figures of the two papers are either very similar or the same. They even have the same numbers. The structures of the two papers are very similar if not the same. The lengths of the two papers are also comparable (18 pages double column in this paper, 20 pages single column in the previous one).

In my understanding, this goes against the policy of journal where, for full papers: «Results previously published at conferences or workshops may be submitted as extended versions.» The previous publication was a journal paper and the extension seems to be very minor.

On page 2, first column, line 32, the authors compare the two papers:
«In a first work [11], we have found some rules to provide relations but with some performance issues. In this article we propose a new approach with SPARQL UPDATE and a spatial triplestore.»
The main novelty, the use of SPARQL UPDATE only reappears in the last paragraph of the paper and no concrete example nor quantified experimental results is provided.

Given the character of this submission that I perceived as abusive, I will not provide a review of its content. My only regret is having spent several hours of my free time (I am not in academia anymore) reading it before checking the previous paper.