In the appendix, the last equation of the proof for theorem 2 (page 16) contains a mistake:

\[\gamma \sup_{s',a',\omega} |\omega^{\top} Q(s',a',\omega') - \omega^{\top} Q'(s',a',\omega’)| = \gamma d(Q, Q').\]

Sadly, this step doesn’t follow the definition of pseudo-metric \(d\) in Equation (17) on page 15, since \(\omega\) and \(\omega'\) could be different at the supreme.

One fixed we propose is to replace \(Q\) with the optimal \(Q^*\), since we’ve shown the existence of a fixed point. In this case, \(\omega\) keeps the same as \(\omega’\), and \(d(Q^*, \mathcal T^nQ') \rightarrow 0\). This could be a weaker version of the original contraction theorem since it may require a small initial \(Q'\).

This corrected theorem is similar to the “pseudocontraction theorem” in Szepesvári and Littman 1999, and in a pseudo-metric space.

Very sorry for the mistake. We sincerely thank the careful readers who pointed out this issue.