Dialogue generation: From imitation learning to inverse reinforcement learning by Ziming Li, Julia Kiseleva, and Maarten de Rijke is online now at this location.
The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator. The reward signal from a poor discriminator can be very sparse and unstable, which may lead the gen- erator to fall into a local optimum or to produce nonsense replies. To alleviate the first problem, we first extend a re- cently proposed adversarial dialogue generation method to an adversarial imitation learning solution. Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator train- ing. We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings. Our experimental results demonstrate that our model can generate more high-quality responses and achieve higher overall performance than the state-of-the-art.
The paper will be presented at AAAI 2019.