Decompovision: Reliability Analysis of Machine Vision Components Through Decomposition and Reuse.
Table of contents:
On this website, we provide the supplementary material for our FSE’2023 submission. The figure below gives an overview of our proposed DecompoVision framework.
Overview
Based on our complex vision task decomposition, we develop a modular reliability framework DecompoVision (shown above), which builds on top of the state of the art reliability framework for image classfication [ICRAF]. Our framework decomposes a complex vision task (c-task) into atomic subtasks (Step I, see Metric Decomposition). Then, for each subtask independently, given an image transformation simulating scene changes, we use human performance data to generate reliability requirements (Step II.a and II.b). Consequently, we compose the individual subtask requirements to get the requirements for the c-task (Step II.c, see Reliability Requirements). Finally, we propose a checking method for both the overall c-task and the subtask requirements, which enables failure localization in the sequence of subtasks. Note that the modularity of Decompovision allows us to reuse human performance data, requirements specifications, and analysis artifacts for shared subtasks across different c-tasks.
This website contains:
Implementation of our framework and instructions for installing and running it: Download DecompoVision.
Definition of compound decomposable metrics and proofs for metrics decomposition: Metric Decomposition.
Experiments to collect human performance data, including instructions and data used in the paper: Human Experiment.
Proofs for correctness theorem: Reliability Requirements.
Additional evaluation results: Additional Evaluation Results.