In the appendix, the last equation of the proof for theorem 2 (page 16) contains a mistake:

$\gamma \sup_{s',a',\omega} |\omega^{\top} Q(s',a',\omega') - \omega^{\top} Q'(s',a',\omega’)| = \gamma d(Q, Q').$

Sadly, this step doesn’t follow the definition of pseudo-metric $$d$$ in Equation (17) on page 15, since $$\omega$$ and $$\omega'$$ could be different at the supreme.

One fixed we propose is to replace $$Q$$ with the optimal $$Q^*$$, since we’ve shown the existence of a fixed point. In this case, $$\omega$$ keeps the same as $$\omega’$$, and $$d(Q^*, \mathcal T^nQ') \rightarrow 0$$. This could be a weaker version of the original contraction theorem since it may require a small initial $$Q'$$.

This corrected theorem is similar to the “pseudocontraction theorem” in Szepesvári and Littman 1999, and in a pseudo-metric space.

Very sorry for the mistake. We sincerely thank the careful readers who pointed out this issue.