jdbcelasticsearchelasticsearch-jdbc-river

jprante elasticsearch jdbc river changing the date value


I am trying to index mysql records in elasticsearch using the jprante's elasticsearch jdbc river. I just noticed that the value in the date field is getting changed in the index.

Mapping:

content_date:{
  "type":"date"
}

content_date field for a record in mysql -> 2012-10-06 02:11:30

after running the jdbc river....

content_date field for same record in elasticsearch -> 2012-10-05T20:41:30Z

River:

curl -XPUT 'localhost:9200/_riv_index/_riv_type/_meta' -d '{
    "type" : "jdbc",
    "jdbc" : {
        "driver" : "com.mysql.jdbc.Driver",
        "url" : "jdbc:mysql://localhost:3306/db",
        "user" : "user",
        "password" : "password",
        "sql" : "select * from table where id=2409",
        "poll" : "1d",
        "versioning" : false
    },
    "index" : {
        "index" : "myindex",
        "type" : "mytype"
    }
}'

Change in date format is acceptable, but why is the date value getting changed? The river is adding utc time difference to the mysql record's date and saving it in elasticsearch. How do I stop this time conversion?


Solution

  • From the Elasticsearch POV, here's what docs said :

    The date type is a special type which maps to JSON string type. It follows a specific format that can be explicitly set. All dates are UTC. Internally, a date maps to a number type long, with the added parsing stage from string to long and from long to string.

    Not sure that you can change it.