Reliability Requirements
In this page, we first provide more details on the parameter estimation method, then to make the page self contain we remind our procedures 1 and 2 used to generate the requirements, next we provide the proof of the Theorem 1, and finally, we show how Procedure 1 can be adapted for compound decomposable evaluation metrics, such as mAP.
Parameter Estimation
To generate reliability requirements for c-tasks, Procedure 1 takes a list of estimated threshold values (
Although not shown in the paper, we also conducted experiments with the transformation brightness and obtained thresholds shown below:
Procedure 1
Procedure 2
Correctness of our requirement composition
Given a c-task
Each req can either be correctness-preservation (cp) or prediction-preservation (
Theorem 1
Theorem 1: If all subtask requirements req
Proof.
Depending on the type of the requirement, req
For
Similarly for
Therefore, req
Procedure for Compound Decomposable Metrics
We provide the following Procedure 3 for generating the reliability requirements for
the c-task
See the table above for examples of generated correctness-preservation requirements with compound decomposable metrics.
For the correctness of the requirement composition using the compound decomposable metrics, we prove the following theorem.
Theorem 2
Theorem 2:
Let
Proof.
Depending on the type of the requirement, req
For
For
Therefore, satisfying all