pythonweb-scrapingscrapyemoji

Scraper - How to save and store emoji

I am currently using Scraper from https://scrapy.org/ under python 3.x and ubuntu environment , somehow i would like to get user comment in forum, and which contain text as well as emoji.

Wondering how we save those emoji into an array so that i can see this in cvs or json ?

Thanks

Solution

  • It will be good if you can at least provide the HTML of the page.

    If emoji and text are enclosed in a parent element, let's say div with comment class like below.

    <div class="comment">
    <div class="description">This is a comment.</div>
    <span>:-)</span>
</div>

    then you can use the following XPath or CSS selector.

    response.css('.comment ::text').extract()

    or 

    response.xpath('.//div[@class="comment"]//text()').extract()