On the Theoretical Convergence and Error Sensitivity Analysis of Yayambo for Fusion of Probabilistic Classifier Outputs

Abstract

This paper considers the problem of classifier fusion for situations where classifier outputs are probabilistic. We study the convergence and analyse the error sensitivity of a performance-agnostic fusion of probabilistic classifier outputs from [1] (called Yayambo). This method is iterative and was proposed for combining probabilistic outputs of black-box classifiers trained for the same task to make a single consensus class prediction. Yayambo considers the diversity between the outputs of the various classifiers, iteratively updating predictions based on their correspondence with other predictions until the predictions converge to a consensus decision or the number of iterations has exceeded a prefixed threshold. For this paper, we address two things. First, as an iterative fusion process, convergence is an ideal way to find consensus [2]. Cases for convergence in the original method was shown experimentally. We establish its conditional theoretical convergence. There may be various reasons to study the convergence of an iterative algorithm, including insurance of obtention of a consensus decision. It should be noted that consensus class does not necessarily mean true class, i.e. Yayambo can converge to a wrong class. Here, the main motivation is the following: investigating on the theoretical convergence of Yayambo might lead into obtaining a closed form expression for the consensus class from the initial probabilities without performing the iteration explicitly. We apply the squeeze theorem [3] to establish convergence of Yayambo and find that Yayambo converges only under certain conditions. Finally, individual classifiers' outputs always contain estimation errors, which can impact the fusion decision. Following [4], [5], we analyse the error sensitivity of Yayambo and find that the error factor can have a dramatic impact on the consensus class. Analysing error sensitivity of a classifier fusion model describes how errors in classifier outputs can (negatively) impact the outcome of the classifier fusion model. This error sensitivity analysis can indicate, for some situations where one needs to select a classifier fusion method for example, which fusion methods would be more appropriate than others.