相關技術也將發表在頂尖會議 WWW 2018 (Cognitive Computing Track). 我們並發現，一般使用者很難區分機器產生或是使用者的留言。
Recently, image captioning has appeared promising, which is expected to widely apply in chatbot area. Yet, “vanilla” sentences, only describing shallow appearances (e.g., types, colors), generated by current works are not satisfied netizen style resulting in lacking engagement with users. Hence, we propose Netizen Style Commenting (NSC), to generate characteristic comments to a user-contributed fashion photo. We are devoted to modulating the comments in a vivid netizen style which reflects the culture in a designated social community and hopes to facilitate more engagement with users. In this work, we design a novel framework that consists of three major components: (1) We construct a large-scale clothing dataset named NetiLook to discover netizen-style comments. (2) We propose three unique measures to estimate the diversity of comments. (3) We bring diversity by marrying topic models with neural networks to make up the insufficiency of conventional image captioning works. The work is also accepted for WWW 2018 (Cognitive Computing Track), also available in arXiv.
To the best of our knowledge, this is the first and the largest netizen-style commenting dataset, NetiLook. It contains 355,205 images from 11,034 users and 5 million associated comments collected from Lookbook, a fashion social media. Most of the images are fashion photos in various angles of views, distinct filters and different styles of collage. Each image is paired with (diverse) user comments, and the average number of comments is 14 per image in our dataset. Besides, each post has a title named by an author, a publishing date and the number of hearts given by other users. Moreover, some users add names, brands, pantone of the clothes, and stores where they bought the clothes. Furthermore, we collect the authors’ public information. Some of them contain age, gender, country and the number of fans. In this paper, we only use the comments and the photos from our dataset. Other attributes can be used to refine the system in future work.
We need to thank the continuing supports from Microsoft Research Asia, the MOST AI Initiatives Projects (科技部), and the fruitful discussions with Dr. Ruihua Song. We also benefit from the grants from NVIDIA and the NVIDIA DGX-1 AI Supercomputer.