I'm using Flutter's Hive package for local storage, and I've been storing data as strings. For instance, take a User
class structured like this:
class User {
final String id;
final String name;
final int age;
final DateTime createdAt;
const User(this.id, this.name, this.age, this.createdAt);
Map<String, dynamic> toJson() {
// Convert object to JSON
}
static User fromJson(Map<String, dynamic> json) {
// Create User from JSON
}
}
When saving a User
instance, I convert it into JSON and save it under a single key:
final user = User('random-id', 'John Doe', 34, DateTime.now());
Hive.box('authenticated_user').put('user', jsonEncode(user));
Similarly for a list of users:
final list = [
User('random-id-1', 'John Doe', 34, DateTime.now()),
User('random-id-2', 'Jane Doe', 30, DateTime.now())
];
Hive.box('users').put('users', jsonEncode(list));
I noticed that Hive also allows storing data as direct object (TypeAdapters). But I'm concerned about the performance and memory usage when choosing between these two storage patterns. So my question is, when reading & writing large amounts of data (e.g. 6000-8000 User
objects) from/to Hive, is it better to store as String or Object (TypeAdapter) and why?
I'd like to understand the pros and cons of both, focusing on:
Performance. (e.g. read/write operations with large data sets)
Memory usage
Data integrity and consistency
Storage size
Hive compaction
I'm not really concerned about:
If i store the data as a String and I open the Memory tab of the Flutter Devtools, I observe a significant memory spike, with usage sometimes exceeding 2GB, when handling around 5000 User
objects. This is much higher than expected, especially since memory usage should ideally remain manageable and not impact app performance negatively.
I also tried separating the code and executing jsonEncode
& jsonDecode
in isolates, and there was some improvements. But again checking the Performance tab of the flutter devtools, i see that the Hive read & write functions still take longer to execute (Self-time > 3secs, Total-time > 4-5secs).
Perhaps it's an issue with the Hive package itself. Maybe hive just wasn't built to handle such large data?
Note: I have not tested the performance when storing data as Objects, i want to know if its beneficial than storing as String
Base on my own experience i would recommend you storing Data as Object (TypeAdapter). Using TypeAdapters can provide enhanced performance, reduced memory consumption, better data integrity, and more efficient storage utilisation. But there are some cons and pros to both:
Data as a String
Pro | Cons |
---|---|
Data in Strings can be easily saved or send to api. | Serialising and deserialising can be inefficient considering the large amounts of data |
Easy in Terms of code | Higher memory consumption |
JSON serialisation/deserialisation could potentially cause errors and data inconsistencies |
Data as Object
Pro | Cons |
---|---|
Storing objects can improve read and write speed, especially with large data. | Less flexibile across platfroms |
Lower Memory Usage | More Complicated in terms of code |
TypeAdapters ensure the data consistency and reduce the risk of serialisation errors | |
Storage Size, it takes up less space in general, because of binary representation. |
Sources
https://www.solutelabs.com/blog/flutter-offline-data-management-hive
https://medium.com/@Ikay_codes/hive-flutter-local-data-storage-with-hive-typeadapter-61e995589e6a