Deep Learning Approaches to Text Production. Shashi Narayan

      2009

      Introduction to Chinese Natural Language Processing

      Kam-Fai Wong, Wenjie Li, Ruifeng Xu, and Zheng-sheng Zhang

      2009

      Introduction to Linguistic Annotation and Text Analytics

      Graham Wilcock

      2009

      Dependency Parsing

      Sandra Kübler, Ryan McDonald, and Joakim Nivre

      2009

      Statistical Language Models for Information Retrieval

      ChengXiang Zhai

      2008

      Copyright © 2020 by Morgan & Claypool

      All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopy, recording, or any other except for brief quotations in printed reviews, without the prior permission of the publisher.

      Deep Learning Approaches to Text Production

      Shashi Narayan and Claire Gardent

       www.morganclaypool.com

      ISBN: 9781681737584 paperback

      ISBN: 9781681737591 ebook

      ISBN: 9781681738215 epub

      ISBN: 9781681737607 hardcover

      DOI 10.2200/S00979ED1V01Y201912HLT044

      A Publication in the Morgan & Claypool Publishers series

       SYNTHESIS LECTURES ON HUMAN LANGUAGE TECHNOLOGIES

      Lecture #44

      Series Editor: Grame Hirst, University of Toronto

      Series ISSN

      Print 1947-4040 Electronic 1947-4059

       Deep Learning Approachesto Text Production

      Shashi Narayan

      University of Edinburgh

      Claire Gardent

      CNRS/LORIA, Nancy

       SYNTHESIS LECTURES ON HUMAN LANGUAGE TECHNOLOGIES #44

       ABSTRACT

      Text production has many applications. It is used, for instance, to generate dialogue turns from dialogue moves, verbalise the content of knowledge bases, or generate English sentences from rich linguistic representations, such as dependency trees or abstract meaning representations. Text production is also at work in text-to-text transformations such as sentence compression, sentence fusion, paraphrasing, sentence (or text) simplification, and text summarisation. This book offers an overview of the fundamentals of neural models for text production. In particular, we elaborate on three main aspects of neural approaches to text production: how sequential decoders learn to generate adequate text, how encoders learn to produce better input representations, and how neural generators account for task-specific objectives. Indeed, each text-production task raises a slightly different challenge (e.g, how to take the dialogue context into account when producing a dialogue turn, how to detect and merge relevant information when summarising a text, or how to produce a well-formed text that correctly captures the information contained in some input data in the case of data-to-text generation). We outline the constraints specific to some of these tasks and examine how existing neural models account for them. More generally, this book considers text-to-text, meaning-to-text, and data-to-text transformations. It aims to provide the audience with a basic knowledge of neural approaches to text production and a roadmap to get them started with the related work. The book is mainly targeted at researchers, graduate students, and industrials interested in text production from different forms of inputs.

       KEYWORDS

      text production, text generation, deep learning, neural networks, meaning-to-text, data-to-text, text-to-text, recurrent neural networks, sequence-to-sequence models, attention, copy, coverage, AMR generation, RDF generation, verbalise, simplification, compression, paraphrasing, dialogue generation, summarisation, content selection, adequacy, input understanding, sentence representation, document representation, communication goals, deep generators, reinforcement learning, evaluation, grammatical, fluent, meaning-preserving, BLEU, ROUGE, relevant, coherent

       To my family.

       – Shashi

       A mes trois rayons de soleil, Jennifer, Gabrielle, et Caroline.

       – Claire

       Contents

       List of Figures

       List of Tables

       Preface

       1Introduction

       1.1What is Text Production?

       1.1.1Generating Text from Meaning Representations

       1.1.2Generating Text from Data

       1.1.3Generating Text from Text

       1.2Roadmap

       1.3What’s Not Covered?

       1.4Our Notations

       PART IBasics

       2Pre-Neural Approaches

       2.1Data-to-Text Generation

       2.2Meaning Representations-to-Text Generation

       2.2.1Grammar-Centric Approaches

       2.2.2Statistical MR-to-Text Generation

       2.3Text-to-Text Generation

       2.3.1Sentence Simplification and Sentence Compression

       2.3.2Document Summarisation

       2.4Summary

       3Deep