Optimization and Evaluation of Reasoning in Probabilistic Description Logic: Towards a Systematic Approach

This paper describes the first steps towards developing a methodology for testing and evaluating the performance of reasoners for the probabilistic description logic P- ${\ensuremath{\mathcal{SHIQ}}(D)}$ . Since it is a new formalism for handling uncertainty in DL ontologies, no such methodology has been proposed. There are no sufficiently large probabilistic ontologies to be used as test suites. In addition, since the reasoning services in P- ${\ensuremath{\mathcal{SHIQ}}(D)}$ are mostly query oriented, there is no single problem (like classification or realization in classical DL) that could be an obvious candidate for benchmarking. All these issues make it hard to evaluate the performance of reasoners, reveal the complexity bottlenecks and assess the value of optimization strategies. This paper addresses these important problems by making the following contributions: First, it describes a probabilistic ontology that has been developed for the real-life domain of breast cancer which poses significant challenges for the state-of-art P- ${\ensuremath{\mathcal{SHIQ}}(D)}$ reasoners. Second, it explains a systematic approach to generating a series of probabilistic reasoning problems that enable evaluation of the reasoning performance and shed light on what makes reasoning in P- ${\ensuremath{\mathcal{SHIQ}}(D)}$ hard in practice. Finally, the paper presents an optimized algorithm for the non-monotonic entailment. Its positive impact on performance is demonstrated using our evaluation methodology.