I'm trying to persist with cygnus using a Mongo sink, data from entities with metadata data estructures. So far, I haven't been able to achieve that.
I'm using cygnus version 0.13.0. Seems to be possible to save metadata info using MySQL and CKAN persistence sinks.
¿Is it possible too using Mongo? ¿Is it configuration matter?
Thanks in advance for any help.
Cygnus does not store the attribute metadata in MongoDB. This is because the internal usage we make of Cygnus when persisting in MongoDB, which imposes a strong constraint regarding this issue.
Anyway, modifying the code in a fork of yourself in order to fix this should be relatively easy. Simply have a look on this method:
private Document createDoc(long recvTimeTs, String entityId, String entityType, String attrName, String attrType, String attrValue) {
Passing an additional parameter String attrMd
and appending this value to the doc
variable should do the trick:
private Document createDoc(long recvTimeTs, String entityId, String entityType, String attrName, String attrType, String attrValue, String attrMd) {
Document doc = new Document("recvTime", new Date(recvTimeTs));
switch (dataModel) {
case DMBYSERVICEPATH:
doc.append("entityId", entityId)
.append("entityType", entityType)
.append("attrName", attrName)
.append("attrType", attrType)
.append("attrValue", attrValue)
.append("attrMd", attrMd);
break;
case DMBYENTITY:
doc.append("attrName", attrName)
.append("attrType", attrType)
.append("attrValue", attrValue)
.append("attrMd", attrMd);
break;
case DMBYATTRIBUTE:
doc.append("attrType", attrType)
.append("attrValue", attrValue)
.append("attrMd", attrMd);
break;
default:
return null; // this will never be reached
} // switch
return doc;
} // createDoc