As answered in this question:
Note that an individual List value in a Cap'n Proto structure has a limit of 2^29-1 elements
Due to this limitation for a single list, I'm trying to split one list that contains more than this amount of items into a map of lists. To do it, I'm using the following schema:
struct Map64UInt{
entries @0: List(Entry);
struct Entry{
key @0: UInt64;
value @1: List(UInt64);
}
}
I've been looking into all examples of the Cap'n Proto but I couldn't find one that contains an example on how to create and add itens into a Capn'Proto List, then add this list into a Cap'n Proto Map. As an example, consider the following code:
void addVecToCapnprotoMap(std::vector<uint64_t> &v){
unsigned int key = 0;
//Here: how do I create a Capn' Proto List of uint64_t
//and add it to a Capn Proto map that follows the schema
//described above?
}
Unfortunately, you cannot dynamically add new elements to a list; you have to specify the full size of the list when you first create it. This is a side effect of Cap'n Proto's zero-copy nature. Since the list is allocated in-place directly into the final message buffer, it can't be resized later.
What you can do instead is maintain a map separately and then write the final list as a last step, like:
// WARNING: Not tested, probably has typos.
class MapHelper {
public:
MapHelper(Map64UInt::Builder finalBuilder)
: finalBuilder(finalBuilder),
orphanage(capnp::Orphanage::getForMessageContaining(finalBuilder)) {}
void add(uint64_t key, const std::vector<unit64_t>& v) {
auto orphan = orphanage.newOrphan<capnp::List<uint64_t>>(v.size());
auto listBuilder = orphan.get();
for (size_t i = 0; i < v.size(); i++) {
listBuilder[i] = v[i];
}
contents.insert(std::make_pair(key, kj::mv(orphan)));
}
void finish() {
// Write all entries to the final map.
auto entriesBuilder = finalBuilder.initEntries(contents.size());
size_t i = 0;
for (auto& entry: contents) {
auto entryBuilder = entriesBuilder[i];
entryBuilder.setKey(entry.first);
entryBuilder.adoptValue(kj::mv(entry.second));
++i;
}
}
private:
Map64UInt::Builder finalBuilder;
capnp::Orphanage orphanage;
std::map<uint64_t, capnp::Orphan<capnp::List<uint64_t>>> contents;
};