We have the context broker configured with cygnus, and the problem that we are getting, is quite strange.
It happen whhen I inserted a new type of element in Context Broker, and we have a subscription to send that element to cosmos. Cygnus send it properly, and create the corresponding hdfs file in cosmos, but when cosmos try to create the Hive table occurs an error.
That is what the cygnus log says.
05 oct 2015 12:51:43,777 INFO [SinkRunner-PollingRunner-DefaultSinkProcessor]
(com.telefonica.iot.cygnus.backends.hdfs.HDFSBackendImpl.provisionHiveTable:225)
- Creating Hive external table=[table_name]
After some hours, we get the following message:
05 oct 2015 18:00:02,489 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor]
(com.telefonica.iot.cygnus.backends.hive.HiveBackend.doCreateTable:77)
- Runtime error (The Hive table cannot be created.
Hive query='create external table [table_name] (columns) row format serde
'org.openx.data.jsonserde.JsonSerDe' location '[hdfs_folder_location]''.
Details=org.apache.thrift.transport.TTransportException)
05 oct 2015 18:00:02,490 WARN [SinkRunner-PollingRunner-DefaultSinkProcessor]
(com.telefonica.iot.cygnus.backends.hdfs.HDFSBackendImpl.provisionHiveTable:237)
- The HiveQL external table could not be created, but Cygnus can continue working... Check your Hive/Shark installation
Between those two events all the actions that should be added in hdfs, they won't be added until we get the error.
Cygnus Version is: 0.8.2
We are using the open cosmos.
This is a well known issue in Cygnus 0.8.2, here you can find a workaround. Nevertheless, this has been fixed in Cygnus 0.9.0 (in order to avoid you have to install it from the sources, I'll upload along the day a RPM to the FIWARE repo).