Aspect-Aware Multimodal Summarization for Chinese E-Commerce Products

Commercial product advertisements, as a critical component of marketing management in e-commerce platforms, aim to attract consumers’ interests and arouse consumers’ desires to purchase the products. However, most product advertisements are so miscellaneous and tedious that the consumers cannot be expected to be patient enough to carefully read through them.

• Paper Link : https://aaai.org/ojs/index.php/AAAI/article/view/6332/6188
• Code : https://github.com/hrlinlp/cepsum
• Dataset : CEPSUM dataset
• Model : MMPG : RAML training, Aspect coverage and coherence, Encoder and Decoder

Contributions

The main contributions are as follows:

• Designed a summarizer that can automatically generate an aspect-aware textual summary for a Chinese ecommerce product by integrating textual and visual product information.
• Proposed a multimodal pointer-generator network and explore various approaches to use product images in the product summarization task.
• Adopted aspect RAML training, aspect coverage, and aspect coherence strategies, which aim to improve importance, non-redundancy, and readability, respectively.
• Constructed a large-scale corpus of Chinese e-commerce product summarization, which includes three subsets that correspond to three categories of products. The experimental results on this dataset demonstrate the superiority of our approach against other methods.

Summary

Proposed aspect-aware summarization model is based on the pointer-generator networks. To incorporate the visual features into the pointer-generator networks, the paper applies three strategies, including initializing the encoder with the global visual feature, initializing the decoder with the global visual feature, and generating context representations with the local visual features. The paper modeled the importance, non-redundancy, and readability of the summary with respect to the product aspects.

• The paper adopted an aspect-based RAML training to generate a summary covering salient product aspects.
• The paper employed an aspect coverage mechanism to eliminate aspect redundancy.
• The paper improved the readability by avoiding aspect rebounding.

The baselines which are compared are :

• Lead
• LexRank
• Seq2Seq
• Pointer-Generator (PG)
• MASS

The metrics used are :

• ROUGE-{1,2,L}