Review Comment:
This manuscript was submitted as 'Application Report' and should be reviewed along the following dimensions: (1) Quality, importance, and impact of the described application (convincing evidence must be provided). (2) Clarity and readability of the describing paper, which shall convey to the reader the key ideas regarding the application of Semantic Web technologies in the application. Please also assess the data file provided by the authors under “Long-term stable URL for resources”. In particular, assess (A) whether the data file is well organized and in particular contains a README file which makes it easy for you to assess the data, (B) whether the provided resources appear to be complete for replication of experiments, and if not, why, (C) whether the chosen repository, if it is not GitHub, Figshare or Zenodo, is appropriate for long-term repository discoverability, and (4) whether the provided data artifacts are complete. Please refer to the reviewer instructions and the FAQ for further information.
The paper describes a forestry interface, the EducaWood, that can be used for educational purposes. The code for the application is on the GitHub repository, however, it does not contain the ontology. There is a SPARQL endpoint at https://crossforest.gsic.uva.es/pruebas/sparql
(1) Quality, importance, and impact of the described application
The application found at https://educawood.gsic.uva.es/ shows a well-developed and pleasant to use tool for forestry domain, which preliminary results demonstrate potential impact in forestry education. Although the interface has a polished design, the paper quality is uncertain due to a couple of drawbacks, as listed below.
1. First and foremost, the EducaWood work seems quite similar to Forest Explorer which has been published in previous papers (see references 38, 41, 51 in the paper). However, the paper does not explain how it advances the contributions made in these papers.
2. The related works section does not include past works similar to EducaWood (papers 41 and 51), and only briefly describes previous work done in paper 38. Moreover, this section does not mention how the described literature is relevant to the paper's work, which limitations were imposed, and which parts this paper adapts, and advances compared to the state-of-the-art. Further, Section 2.2 is a mix of different elements (visualizations, REST APIs etc) making it difficult to follow and understand their connection to the paper contributions; I would recommend rewriting this section or spliting it to more sections each with a distinctive content.
3. As the paper focuses on the EducaWood interface, it lacks the clear connection and advancement to the semantic web, especially since the architecture of EducaWood, and the creation of annotations seem very close to those of Forest Explorer. Consequentially, I would recommend clearly stating the differences of the current paper output to previously published works. Also, it would be helpful adding in the Table 2 dependencies related to SW. Moreover, the paper can be benefited of a section describing the underlying ontology and knowledge graph in numbers, and comparing them with state-of-the-art resources and the user-interface connections they provide.
4. I am not sure I understand well Table 1, as I would expect having the namespaces used in the ontology, however I am not certain this is the case as no information is further provided in the text. Moreover, the namespaces used raises concerns about best practices for ontology development as the same namespace (http://educawood.gsic.uva.es) is reused under different prefixes.
5. Highlighting the pedagogical aspect of the special issue, I would expect more details about:
5.1. which aspects of the system and interface are aligned with which pedagogical and learning settings. Currently, the paper's discussion of educational aspects takes place only at the end of the paper (Section 4 and 5). This leads to plenty of unsupported claims, about the applicability of EducaWood in different educational levels, learning objectives, learners interest, interdisciplinary learning, collaborative learning, and logical awareness, because it misses any connection with the EducaWood system and interface.
5.2. Section 4.2, regarding the experiment, such as the students background knowledge checks (if any), students introduction to EducaWood steps, an example of the annotation session and verification, more details about the design and learning objectives of the experiment and the choice of different steps (min 20 trees, forestry management) motivating the usage of SUS questionnaire, what type of teachers feedback was provided, which was the alternative paper-pencil activity and how EducaWood is making thinks better/easier, qualitative analysis and presentation of the students annotations (in comparisson with their scores) and discussion of minor bugs and how they might have affected the UX.
6. Regarding the educational contribution, the results of SUS (75% with s.d. 11.5) seem very similar to Forest Explorer SUS results (75% with s.d. 16 in general. If the two systems are similar (to which degree is left to the revision to be clarified), I would need to see how the EducaWood is better since the SUS results do not demonstrate any potential for statistical significance.
7. Regarding the design of web applications, the paper claims to contribution to the "good practices", but it is unclear which are those and how they apply them in EducaWood.
(2) Clarity and readability of the describing paper
The presentation and readability could be improved in most of the sections.
I would recommend having a different 1st paragraph in the Introduction that is more related to the topic.
I would highly encourage restructuring and editing the paper so it clearly explains the problem, proposed solution and contributions of the current work compared to the literature.
Minor comments:
- kindly avoid the ...
- adding "The" before the namespace:name in the beginning of a sentence
- make footnotes before ; ie ZOOMz;\footnote{} -> ZOOMz\footnote{};
- adding limitations to Discussion section
- reformulating the Discussion section to highlight the key findings and contributions of the paper
- adding a separate future work and conclusion section for the summary of the paper
- kindly make the Figure 1 with transparent background, if possible
2 (A) the data file is well organized and in particular contains a README file which makes it easy for you to assess the data
yes
2 (B) whether the provided resources appear to be complete for replication of experiments,
no, the user study data are not provided
2 (C) whether the chosen repository is appropriate for long-term repository discoverability,
yes
(4) whether the provided data artifacts are complete.
There is a GitHub does not connect to the underlying ontology neither to the SPARQL endpoint.
Questions for the authors:
1. In the description of Figure 4b, why is the taxon information obtained from DBpedia and Wikidata, and not the underlying ontology and KG?
2. In Listing 1, why most of the IDs are the same (the "Neik7P0woiD")?
|
