NIS2Onto: an Ontological Representation of the NIS 2 Directive

In this paper, the authors propose an ontology to model the NIS 2 Directive of the European Union. In times of increasing legal complexity support to cope with the legal requirements and ensure compliance with is highly welcomed. Additionally, the topic of the paper also fits into the scope of the journal.
While the structure of the paper is suitable for the content, it would be highly beneficial for the reader to include more information. I understand that the paper is an extension of previous work, however, given the title of this paper users would expect to get all the information about the proposed ontology without having to read additional papers. Additionally, the previous paper contains more examples and figures from which this paper would also benefit. I would like to ask the authors to explicitly state what is new in this paper compared to the previous work(s).

In general, the paper is written in an understandable way and easy to follow, however, the writing style needs to be improved: There are many occurrences in the paper which are not precise enough and could be improved by adding information. For instance, missing references (e.g., line 38: AI Act) or use of abbreviations without prior spelling of the full name (NVD, GDPR, ISAP, PCI DSS,…). Furthermore, there are statements throughout the paper where adding the concrete information should be easy and would also help readers in understanding, some examples:
• “issued less recently” -> add the date
• “most important sectors” -> which sectors are important, who says that they are important?
• “in some cases” -> which ones?
• “a small part” -> of what?
• “some security best practices”
• “somewhat overlaps”

Additionally, the proposed ontology is wrongly spelled multiple times throughout the paper.
Since it is a directive, EU member states need to transpose it into national law. I’d like to ask the authors to elaborate on the impact of different national transpositions on the proposed ontology.
The proposed ontology is provided via a Github repository. I didn’t find the link to the repository in the paper. There is no additional information available in the repository, there is also no readme available.

The authors follow the SecOnto methodology to create the ontology without providing details. While there approaches mentioned that did not seem to succeed, it is only stated that the ontology has been created “semi-manually”. The paper would therefore benefit from a detailed description of how the ontology has been created, including concrete examples of how the legal text is translated into classes and properties. Please also elaborate on the reuse of existing ontologies. Furthermore, nine different types of competency questions are defined in Section 3.4 but there are no actual competency questions. I’d like to ask the authors to add competency questions, which can also be used and answered in the ontology evaluation section.

The provided ontology itself (OWL file) does not contain meta information (e.g., authors, contributors, license, version,…) and also no labels or descriptions of the classes or properties. Also, there is not even a dedicated ontology IRI available, instead the standard IRI provided by Protégé is used. It would be very beneficial to register a namespace for the ontology (e.g., w3id.org). Where are the SWRL rules stated? Please also elaborate on the maintenance plan of the ontology. The evaluation of the ontology is done by providing metrics. I would expect an ontology evaluation to also contain the answered competency questions to show that the modelled ontology can answer the questions. Furthermore, I am missing an evaluation according to the FAIR principles (e.g., https://foops.linkeddata.es/FAIR_validator.html) which would also give a hint on the reusability of the ontology for interested parties. Could you please also elaborate on how you see the future application of the proposed ontology? For the case study, please add concrete examples to the paper as well as the SPARQL query.

Overall, the paper addresses an interesting topic and the proposed ontology could be interesting for real-world applications. However, the paper does not provide the level of details expected of a journal, the writing style is not precise enough and the proposed ontology and its evaluation is missing parts. Therefore, this paper is not ready for publication in its current state and I suggest a major revision.

Paper Summary:
In this paper, the authors describe the development of an ontology that conceptualizes the requirements of the NIS 2 directive, and provide concrete example on how a company can use it to track which requirements they are already in compliance with and which ones they are still non-compliant.

Evaluation as an ‘Ontology description’ manuscript:
(1) Quality and relevance of the described ontology: The work is appropriate for this journal, however, it is difficult to assess its quality due to a short evaluation (I give pointers on how to improve it further on) and due to a lack of human-readable documentation for the ontology itself (the provided repository only has a .owl file). A proper state of the art is also missing, in particular related to ontology-based regulatory compliance.

(2) Illustration, clarity and readability of the paper: The paper could use a thorough review to have a more formal style of writing. Moreover, as I elaborate in the next sections of this review, it lacks figures and examples that assist with the overall readability of such a contribution.

(3) Long-term stable URL for resources: The organisation of the repository should be improved as the provided URL does not contain a README file to guide readers through the contents of the repository. The provided OWL ontology appears to be complete for the replication of experiments and is stored on GitHub for long-term preservation and discoverability. There is no provenance information related to the contents of the repository. There is no human-readable documentation for the ontology. The versioning strategy to be used by the authors should also be mentioned.

Detailed evaluation:
Title & Abstract. The title is well-drafted and presents the main goal of the manuscript. The abstract describes well enough its purpose, findings and value, however the methodology used to create the ontology is not mentioned. The abstract should also provide an insight on how much coverage of the directive is provided in the ontology, as well as situate it in relation to the state of the art. The authors also mention that “NIS2Onto reduces time and effort required for the verification, both by enabling continuous monitoring of any modification that occurs during the lifespan of the company and by minimising the risk of error by human personnel”, but do not further elaborate on how they ensure this in any section of the paper, e.g., to establish that using their solution is more time-effiecient, I would expect a user-study to be performed and assessed.

1. Introduction. This section introduces the motivation of the paper. Although the ontological contribution of the work is well covered – the development of an ontology to describe the NIS2 directive –, it is not clear if/how the work was validated by legal experts or how the authors will support the claims of having an adaptable and enforceable compliance mechanism that is interoperable across implementers, which should also be a contribution of the manuscript if the ontology is to be used for automated compliance verification. The second paragraph should properly introduce the scope of NIS2 and to which entities it applies, and also compare the differences between NIS2 and its predecessor. Moreover, this section should also be supported by valid references, including citations for the laws and directives, and improved with definitions from the domain, to help readers that are not experts in cybersecurity and its regulatory aspects. Limitations, if any, are absent from the introduction or the overall paper.

2. Related work. DPV [1] is a state of the art resource to represent information related to usage and processing of personal data, which includes an extension to support the NIS2 Directive [2], which is currently missing from this Related Work section. For completeness, contributions related to cybersecurity ontologies [3-6] (non-exhaustive list) can also be considered for integration in this section, as well as to check whether they are good extension points for the developed ontology. There is also an extensive body of work related to ontology-based regulatory compliance, which is completely missing from this contribution, including using SWRL rules as proposed by the authors. This analysis will be helpful to understand why the authors chose SWRL for their work.

3.1 Overview of the NIS 2 Directive. The author’s decision to not consider Chapter I for the ontology is not comprehensible given that it includes information about the scope, involved entities and definitions that support the Directive’s legislative text. Moreover, a diagram would help readers to visualise the involved NIS2 entities and how they are expected to interact due to the obligations of the directive. A proper introduction to each studied Chapter and its requirements will also help the readers to understand the complexity of NIS2.

3.2 SecOnto Methodology. The contribution would benefit from a diagram/figure explaining the followed methodology, as it is an adapted version of Methontology for the security domain. Such a figure would also help connecting the dots between the proposed methodology and the following subsections related to the development of the ontology (subsections 3.3 to 3.10). The provided description is also not very clear regarding the outcomes of each step of the methodology, e.g., the Implementation step has the ontology, ‘associated paperwork’, and ‘conclusions drawn from the ontology’ as outcomes — what paperwork or conclusions are expected here? In what format? The followed methodology also does not mention any step of legal validation. Other ontology engineering methodologies, such as LOT [7], do this by integrating domain experts, e.g., in this case legal experts, in all phases of ontology development. A couple of sentences can also be added to address now NIS2Onto will adapt to the Member States national implementations of the directive.

3.3. Automating the Creation of the Ontology. The full NLP pipeline to automatically extract concepts from the directive should be better detaileed. In particular, the results of the extraction and its accuracy should be further elaborated by the authors, to provide an understanding of the efficiency, correctness and completeness of the chosen methodology. The protocol for manual evaluation of the created concepts is also missing from this section’s description. The authors also mention that “We deliberately avoided using certain tools, particularly in the context of ontological automated development, because they proved to be ineffective or incompatible with the context of security directives.”. Which tools are included here and why are they not eefective or compatible with security directives?

3.4. Competency Questions. The Compliance check CQ should be split in 2: one for checking whether an entity is compliant and a second one for checking if they are compliant with a certain article. The Integration I and Integration and Differential Analysis CQs introduce integration with other regulations, however this aspect has been left out of the manuscript so far. I would recommend to address this interplay of regulations in a previous section, perhaps in the section decribing the directive. Overall, the CQs would benefit from a more formal structuring.

3.5. NIS2Onto Overview. This section would benefit from a schematic diagram to showcase the main classes and properties of NIS2Onto. Moreover, the authors mention that NIS2Onto “associates specific agents with the security measures the agents must fulfil through the equivalence relationship, i.e., EquivalentTo” — I would expect that an equivalence relationship is used between related elements and not between distinct concepts such as entities, e.g., agents, and security measures. Furthermore, the usage of the term ‘agents’ is ambiguous and must be introduced, probably even in an earlier section. Are agents natural persons, legal entities, software agents, others? An example instantiation of a security measure and its associated actions and entities would also increase the understandability of this section. Concretely, the role of reasoning to evaluate generated inferences should be properly introduced and explained — what additional knowledge does it provide over the already extracted knowledge base? Providing an example of an instantiation of a measure into the concrete adoption of a specific standard would also give further readability to the text.

3.6. Classes and individuals. Examples of documents, agents and objects can be provided in this section. Furthermore, objects are very abstract classes that are seemingly trying to cover different concepts within the same class — if possible, dividing them and using separate classes for different concepts with be ideal. The modelling of compliance classes is not clear. Taking the used example, an instance of Article-10-MemberState-Compliant is supposed to be a subclass of all classes related to Article 10 that a Member State has to comply with or is it just a subclass of the ones which it is already compliant? The text seems to imply the former, while the term itself, i.e., Article-10-MemberState-Compliant, by using the word ‘Compliant’, seems to imply the latter. If what the authors which to express is the former, I would suggest changing the naming of the classes to use the word ‘Compliance’ instead and then use terms like ‘Compliant’ or ‘Non-Compliant’ to express the status of compliance of each class that needs to be complied with for a certain article/entity.

3.7. Object-properties (also applies to 3.8. Data-properties). Similar comment in the previous section, as the authors mention ‘The object property name is obtained by the verb, but it includes other additional elements.’. The result are complex terms that will hinder reusage, e.g., for other security-related ontologies.

3.9. SWRL rules. As mentioned in the comments related to Section 2, there is no justification on why the authors decided to use SWRL as their reasoning language, and as such, no state of the art on existing solutions for regulatory compliance, in particular for security, using SWRL or any other languages. Furthermore, considering that rules are used to verify compliance with the regulation, it would be good to understand their coverage of the regulation, e.g., if there is a rule for each measure in NIS 2, and the rules should be provided together with the ontological resource, e.g., in the linked code repository.

3.10. Evaluation. The provided evaluation can be further improved by the usage of tools such as OOPS! [8] and FOOPS! [9] to access if there are critical pitfalls in the ontology development and whether it follows the FAIR principles for ontology publication. It is also not clear how far the ontology goes in terms of answering the defined competency questions, e.g., SPARQL queries for all competency questions can be provided.

5. Conclusions. The authors mention that ‘NIS2Onto […] supports risk assessment,’ — this sentence should be supported in the previous sections. A concrete suggestion would be to improve the case study section with a risk assessment that is concretly supported by the use of NIS2Onto. As future work, I would also invite the authors to look into how to integrate their work with DPV, as it is a state of the art vocabulary for data protection-related requirements, which is also looking to have NIS 2 as one of its extensions. As such, I am at the disposal of the authors in case they want to work on such an integration, as I am an active member of the Community Group [10] that edits and maintains the DPV.

Minor comments:
Capitalise mentions of ‘Chapters’ and ‘Articles’ in the manuscript and use it consistently.

3.2 SecOnto Methodology
Page 4 Line 7: ‘the ontology that describes the measurements’ -> measures ?
Page 4 Line 15-16: ‘composed of the articles from 7 to 37’ -> composed by Articles 7 to 37

3.3. Automating the Creation of the Ontology
Page 4 Line 35: ‘of SpaCy and ClausIE library’ -> of the SpaCy and ClausIE libraries

3.6. Classes and individuals
NIS2Onto is mispelled in page 6, lines 13 an 14
Page 6 Line 28: ‘The class names adopted has been obtained’ -> have been

4. Case study
Page 10 Line 21: ‘these are Article 12, paragraphs 1 and 2’ -> Figure 3 mentions Article 21, not 12

[1] https://arxiv.org/abs/2404.13426
[2] https://w3id.org/dpv/legal/eu/nis2
[3] https://link.springer.com/chapter/10.1007/978-3-030-63479-7_22
[4] https://ieeexplore.ieee.org/abstract/document/8205615
[5] https://www.mdpi.com/1424-8220/18/9/3053
[6] https://link.springer.com/chapter/10.1007/978-3-319-98842-9_1
[7] https://lot.linkeddata.es
[8] https://oops.linkeddata.es
[9] https://foops.linkeddata.es/FAIR_validator.html
[10] https://www.w3.org/community/dpvcg/

The stated objectives of the work are to develop a NIS2 ontology that represents entities, relationships, and obligations, and using these for automated compliance verification and risk assessments. The artefact being presented is an OWL2 ontology, which is available at the given url. Overall, the work is extremely timely as the NIS2 implementation is kicking off at the moment, and there are specific policy and governance measures being adopted in response. However, the writing of the paper is severely lacking in describing the rationale and methodology for how the ontology was developed, assessment of the quality of the ontology, and how it is being applied to achieve the stated goals. At the same time, it is clear that the authors have performed a large amount of work as the artefact demonstrates a large number of concepts with terminology aligned to the NIS2 contents, though there also exists prior work by the same author which has significant overlaps in terms of methodologies and explanations of how to develop ontologies. It seems that the only contribution of this article is a quantification of that existing knowledge in the form of an ontology, and raises questions of novelty without sufficient evidence. Due to this, I think the article as well as the ontology/resource requires a large amount of work in terms of communicating the various aspects of its development and application, as well as regarding the application of best practices and evaluation. In its present state, I do not recommend publishing this article.

## Comparison with Prior Work by Authors

- The reference [2] is mentioned as prior work on pg.2:16 as "extends a previous contribution ... where the ontological approach ... is just sketched out" - what is the different in the two works is not further clarified. The ontological approach mentioned in the previous paper seems more detailed than the rudimentary description of the work presented in this paper.
- If the previous paper was used as the methodology to create the ontology, and it provides sufficient information - then this should be explicitly stated, and a summary should be included in this work
- The SecOnto methodology in [5], also be the same author(s) is explained to have been used in Sec.3.2 - however it is not explained how that method was used to specifically construct the ontology.
- In continuation of above, how does this methodology compare with other methodologies used in the legal domain? OR even general methods like Linked Open Terms and its predecessor NeOn?
- The SecOnto paper [5], now published as https://doi.org/10.1016/j.cose.2024.104150 in Computer & Security journal, has a large overlap with this article in terms of describing in great detail how each article and clause was used to create a specific concept and how that is being used for developing the ontology - so the question arises as to what additional value does this paper provide in comparison to that (already complete and published) work?
- In Sec 3.3, the authors mention semi-automatically generating the ontology - again how was this done is not explained in detail. What concepts were extracted through this process, how was it ensured that there were no errors in interpretation? For an ontology with a stated purpose of legal interpretation and compliance, any lapse or error can be of serious consequence. And at the same time, if this approach was based on the prior works as outlined in earlier comments - the paper should mention this clearly in terms of how it was applied and what is the distinction between this and the earlier works.

## State of the Art

- in the introduction, it would be good to briefly explain why you have chosen semantic web to model this information, in particular what role do ontologies play in your specific motivation?
- in the introduction as well as elsewhere, you have made a choice to only consider work explicitly modelling cybersecurity requirements, and in particular specific works related to modelling GDPR - these leave out a wide variety of other approaches that have modelled legislations (e.g. LegalRuleML, GDPRtEXT), or have developed ontologies specific to NIS2 (DPV 2.0 https://w3id.org/dpv/legal/eu/nis2), or developed approaches to evaluate compliance (e.g. Analysis of ontologies and policy languages to represent information flows in GDPR https://doi.org/10.3233/SW-223009).
- Given that there is a significant amount of work existing regarding the implementation of GDPR, not comparing or considering how that work influences or compares with your approach is IMO a significant drawback
- The references in Section 2 are mentioned without any context or relation back to the proposed work or its objectives e.g. PrOnto is mentioned as an ontology for GDPR - but that ontology is from 2018, is not available to be reused, and doesn't specifically model the security aspects required by GDPR. Similarly, other ontologies mentioned are also from 2018 - which is now 6 years in the past - and no statements are made as to whether these are reusable, available, or even if you looked at them and were influenced by their modelling and/or concepts.
- There is a large amount of work being published by ENISA regarding the implementation of the NIS2 requirements - how does this work compare or integrate or plan to use that?

## Ontology Engineering

- Sec 3.4 is titled competency questions, however the list provided is for functionalities and applications of the ontology rather than a list of questions that was used to determine the concepts and modelling within the ontology
- It is never explained in the article what the concepts are supposed to represent specifically within NIS2 - there are clauses being represented, specific concepts for entities, actions, documents - but they are all bundled together
- The assocaited artefact at https://github.com/gianpietroc/nis-ontology/blob/main/NIS2Onto.owl is accessible, and shows further issues in being unclear as to what is being modelled, there are far too many confusing modelling implications
- For example, in the ontology, Incident is a class, IncidentImpact is a subclass of Incident - which is quite weird if we think that the impacts of incidents are also being considered impacts instead of being a separate concept on its own. As NIS2 as specific obligations regarding incidents, this leads to a recursive loop where resolution of an incident results in more incidents are impacts are identified or detected and are also treated as incidents. Regulations therefore always distinguish between incidents and impacts in their obligations, which the ontology doesn't seem to do.
- Another example, there are a large number of cases where a class is defined as an instance of itself, such as IncidentImpact is declared as a type of owl:Class as well as an IncidentImpact (self). This results in recursive loops which will render any reasoning unfeasible and impossible. How this was not detected or what choice of modelling led to the creation of such a design pattern? Why is this not mentioned in the article?
- The concepts in the ontology have no annotations to support human understanding and usability - there are no labels, no comments, no documentation whatsoever. The github repo also does not contain any documentation.

## Lack of Evidence of Utility

- there are several statements made in the article regarding the utility of the produced ontology, such as pg.2:11 "these requirements can change over time and such changes are handled automatically" - which in the context of a regulation is a very influential statement to make as the regulations have implementing legislations, case law, and other implementation details, especially as this is a directive, and it is entirely unclear how the ontology plans to automatically keep itself updated for that. Similarly, the claim pg.2:9 "fewer resources are required for compliance verification" is not justified with evidence in the article.
- pg.8 describes the application of ontology to a specific use-case, however the concepts are directly shown as being used in a rule without an explanation of how that rule was constructed and how dose the ontology help compliance processes
- There is a risk here that the ontology is being too literal in representing the text of the regulation directly, and will require a large amount of effort as the interpretation of the law evolves and additional considerations are identified - which means the ontology will have to be updated with further concepts and properties to model these. How the existing rules can incorporate that practicality is not considered or even discussed.
- Sec.3.10 is titled evaluation, but no actual evaluation is shown. There are some metrics, and three tasks listed without any information on how they were carried out or what was the outcome.
- There is no reference to existing best practice e.g. OOPS! or FOOPS! for evaluation, WIDOCO best practices for documentation, or even the use of a semantic reasoning step for ensuring logical consistency
- Sec.4 describes a case study which directly states the modelling of a specific rule without offering any insight into how someone who doesn't know the ontolgy will approach the task. Combined with a lack of information or documentation, it is difficult to see how such a case study will be implemented in practice.

## Style / Formatting

- pg.1:48 it would be better to provide the url in the main text or as a footnote, rather than as a reference which is intended more to communicate resources external to the work being presented
- pg.2:1-6 the phrasing here is confusing and it is not clear as to what is being stated - please clarify and write more directly for a reader not familiar with your style of modelling or your interpretations of the processes involved in use of the ontology

## Declarations

- GDPRtEXT and DPV are works where I am an involved author, and they are referenced in context of their status as state of the art.
- The DPVCG, which maintains the DPV, is working on the NIS2 extension for modelling concepts relevant to the implementation of the NIS2 regulation. DPV already models GDPR, AI Act, and other regulations, and provides a framework to create harmonised information systems for legal compliance related to data and AI laws. The authors are invited to consider analysing the DPV as part of the state of the art, and to participate and contribute their work on NIS2 for the DPV's next iteration.