Not all images are interesting to people. People are drawn by interesting images and ignore tasteless ones. Image interestingness has the importance no less than other subjective image properties that have received significant research interest, but has not been systematically studied before. In this work we focus on visual and social aspects of image interestingness. We rely on crowdsourcing tools to survey human perceptions for these subjective properties and verify data by analyzing consistency and reliability. We show that people have an agreement when deciding if an image is interesting or not. We examine the correlation between the social, visual aspects of interestingness and aesthetics. By exploring the correlation, we find that: (1) Weak correlation between social interestingness and both of visual interestingness and image aesthetics indicates that the images frequently re-shared by people are not necessarily aesthetic or visually interesting. (2) High correlation between image aesthetics and visual interestingness implies aesthetic images are more likely to be visually interesting to people. Then we wonder what features of an image lead to social interestingness, e.g. receiving more likes and shares on social networking sites? We train classifiers to predict visual and social interestingness and investigate the contribution from different image features. We find that social and visual interestingness can be best predicted with color and texture, respectively, providing a way to manipulate social and visual liking of images with image features. Further, we investigate the correlation between social/visual image interestingness and image color. We find that colors with arousal effect show more frequently in images with higher social interestingness. That could be explained by previous studies for activation-related affect of colors and provides useful and important advice when advertising on social networking sites.