Certainly. There is already research being done on developing deep learning / convolutional neural networks to do exactly this. Four recent references as of January 2018 are given below.

The main challenges with doing it are:

Acquiring a large enough dataset (human face images and their respective attractiveness scores) with proper subject approval.
The fact that attractiveness is subjective and varies with ethnic group and culture. Therefore such training data will have a broader range of labels than in more classical recognition tasks such as object detection (for which the label is binary), leading to more uncertainty in the network's predictions. For this reason most research focuses on training networks for a specific group.

This research area isn't being developed hugely (at least in academia) at the moment most likely because of ethical considerations with acquiring such sensitive data and dubious uses. I suspect that now companies like OKCupid and Match.com are or will be developing this research privately for the purposes of automatic match making.

Xu et al., A new humanlike facial attractiveness predictor with cascaded fine-tuning deep learning model, arXiv 2015, paper

Gan et al., Deep self-taught learning for facial beauty prediction, Neurocomputing 2014 paper

Wang et al., Attractive or Not?: Beauty Prediction with Attractiveness-Aware Encoders and Robust Late Fusion, ACM international conference on Multimedia 2014 paper

Shen et al., Fooling Neural Networks in Face Attractiveness Evaluation: Adversarial Examples with High Attractiveness Score But Low Subjective Score Multimedia Big Data (BigMM), 2017 IEEE Third International Conference on paper