Is there a way to persist instances of a class in memory or file system in Python? Can I do this with shelve?
The following line is part of this tutorial which takes a long time to execute, and I need to cache it for next program executions.
clf = MultinomialNB().fit(X_train_counts, training_data['targets'])
Type of clf:
>>> type(clf)
<class 'sklearn.naive_bayes.MultinomialNB'>
Yes, you can use shelve to persist instances of a class. shelve gives you a dictionary interface, making the process relatively transparent.
Underneath, shelve uses the pickle library; if the shelve API doesn't fit your needs, you can go straight to that module.
scikit-learn explicitly support pickle, see Model persistence:
After training a scikit-learn model, it is desirable to have a way to persist the model for future use without having to retrain. The following section gives you an example of how to persist a model with pickle.