Xiaojun WAN. Intelligent text generation: recent advances and challenges[J]. Big data research, 2023, 9(2): 99-109.
DOI:
Xiaojun WAN. Intelligent text generation: recent advances and challenges[J]. Big data research, 2023, 9(2): 99-109. DOI: 10.11959/j.issn.2096-0271.2023014.
Intelligent text generation: recent advances and challenges
Intelligent text generation is one of the advanced research directions in the fields of artificial intelligence and natural language processing.It is one of key technologies for making AIGC successful
and it has been highly concerned by both academia and industry in recent years.The technology has been deployed and used in many application areas including media publishing and e-commerce
and it can much improve the efficiency of text content production.The systematic overview of the applications of intelligent text generation and the mainstream ways of text generation were given.And the recent deep learning techniques for text generation were introduced.Lastly
the challenges faced by neural text generation were summarized.
关键词
Keywords
references
REITER E , . An architecture for data-totext systems [C ] // Proceedings of the 11th European Workshop on Natural Language Generation . New York:ACM Press , 2007 : 97 - 104 .
SUTSKEVER I , VINYALS O , LE Q V . Sequence to sequence learning with neural networks [C ] // Proceedings of the 27th International Conference on Neural Information Processing Systems . New York:ACM Press , 2014 : 3104 - 3112 .
VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York:ACM Press , 2017 : 6000 - 6010 .
KINGMA D P , WELLING M . An introduction to variational autoencoders [J ] . Foundations and Trends® in Machine Learning , 2019 , 12 ( 4 ): 307 - 392 .
GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J ] . Communications of the ACM , 2020 , 63 ( 11 ): 139 - 144 .
LI X L , THICKSTUN J , GULRAJANI I , et al . Diffusion-LM improves controllable text generation [J ] . arXiv preprint , 2022 ,arXiv:2205.14217.
DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [J ] . arXiv preprint , 2018 ,arXiv:1810.04805.
LIU Y H , OTT M , GOYAL N , et al . RoBERTa:a robustly optimized BERT pretraining approach [J ] . arXiv preprint , 2019 ,arXiv:1907.11692.
RADFORD A , NARASIMHAN K , Salimans T , et al . Improving language understanding by generative pretraining [Z ] . 2018 .
RADFORD A , WU J , CHILD R , et al . Language models are unsupervised multitask learners [J ] . OpenAI Blog , 2019 , 1 ( 8 ).
BROWN T B , MANN B , RYDER N , et al . Language models are few-shot learners [C ] // Proceedings of the 34th International Conference on Neural Information Processing Systems . New York:ACM Press , 2020 : 1877 - 1901 .
OUYANG L , WU J , JIANG X , et al . Training language models to follow instructions with human feedback [J ] . arXiv preprint , 2022 ,arXiv:2203.02155.
LEWIS M , LIU Y H , GOYAL N , et al . BART:denoising sequence-to-sequence pretraining for natural language generation,translation,and comprehension [C ] // Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg:Association for Computational Linguistics , 2020 : 7871 - 7880 .
COLIN R , NOAM S , ADAM R , et al . Exploring the limits of transfer learning with a unified text-to-text transformer [J ] . Journal of Machine Learning Research , 2020 , 21 : 5485 - 5551 .
LIU A , SAP M , LU X M , et al . DExperts:decoding-time controlled text generation with experts and anti-experts [J ] . arXiv preprint , 2021 ,arXiv:2105.03023.