shodan

Output json file on some fields without filtering data with Shodan?


I've downloaded some JSON data from Shodan, and only want to retain some fields from it. To explore what I want, I'm running the following, which works-

shodan parse --fields ip,port --separator , "data.json.gz"

However, I now want to output/ export the data; I'm trying to run the following -

shodan parse --fields ip,port -O "data_processed.json.gz" "data.json.gz"

It's requiring me to specify a filter parameter, which I don't need. If I do add an empty filter as so, it tells me data_processes.json.gz doesn't exist.

shodan parse --fields ip,port -f -O "data_processed.json.gz" "data.json.gz"

I'm a bit stumped on how to export only certain fields of my data; how do I go about doing so?


Solution

  • If you only want to output those 2 properties then you can simply pipe them to a file:

    shodan parse --fields ip,port --separator , data.json.gz > data_processed.csv
    

    A few things to keep in mind:

    1. You probably want to export the ip_str property as it's a more user-friendly version of the IP address. The ip property is a numeric version of the IP address and aimed at users storing the information in a database.
    2. You can convert your data file into Excel or CSV format using the shodan convert command. For example: shodan convert data.json.gz csv See here for a quick guide: https://help.shodan.io/guides/how-to-convert-to-excel