Why does the mutex lock twice in LevelDB?

I'm learning the levelDB C++ project, here is the situation, Status s = Write(WriteOptions(), nullptr) triggers a compaction work, and then enter the while loop to wait the signal, the compaction thread goes to BackgroundCall method, and it also needs to lock the mutex_, I'm not sure the TEST_CompactMemTable still holds the mutex, so I print the debug message in the Lock and Unlock method. The output is just like:

TEST_CompactMemTable mutex lock
BackgroundCallmutex lock
BackgroundCallmutex unlock
TEST_CompactMemTable mutex unlock

I'm confused why does the mutex lock twice? Am i missing something, any help would be greatly appreciated.


Status DBImpl::TEST_CompactMemTable() {
  Status s = Write(WriteOptions(), nullptr);
  if (s.ok()) {
    // lock the mutex first
    MutexLock l(&mutex_);
    while (imm_ != nullptr && bg_error_.ok()) {
      background_work_finished_signal_.Wait();
    }
    if (imm_ != nullptr) {
      s = bg_error_;
    }
  }
  return s;
}

void DBImpl::BackgroundCall() {
  // lock the mutex twice
  MutexLock l(&mutex_);
  assert(background_compaction_scheduled_);
  if (shutting_down_.load(std::memory_order_acquire)) {
  } else if (!bg_error_.ok()) {
  } else {
    BackgroundCompaction();
  }

  background_compaction_scheduled_ = false;

  MaybeScheduleCompaction();
  background_work_finished_signal_.SignalAll();
}

Solution

This is simple background_work_finished_signal_ is conditional variable which is associated with a mutex_.

This is done during construction: see DBImpl::DBImpl

DBImpl::DBImpl(const Options& raw_options, const std::string& dbname)
    : env_(raw_options.env),
      internal_comparator_(raw_options.comparator),
      internal_filter_policy_(raw_options.filter_policy),
      options_(SanitizeOptions(dbname, &internal_comparator_,
                               &internal_filter_policy_, raw_options)),
      owns_info_log_(options_.info_log != raw_options.info_log),
      owns_cache_(options_.block_cache != raw_options.block_cache),
      dbname_(dbname),
      table_cache_(new TableCache(dbname_, options_, TableCacheSize(options_))),
      db_lock_(nullptr),
      shutting_down_(false),
      background_work_finished_signal_(&mutex_),
      mem_(nullptr),
      imm_(nullptr),
      has_imm_(false),
      logfile_(nullptr),
      logfile_number_(0),
      log_(nullptr),
      seed_(0),
      tmp_batch_(new WriteBatch),
      background_compaction_scheduled_(false),
      manual_compaction_(nullptr),
      versions_(new VersionSet(dbname_, &options_, table_cache_,
                               &internal_comparator_)) {}

Now when background_work_finished_signal_.Wait(); is called mutex have to be released, so other thread can lock it and send notifications. When notification is received lock is restored before background_work_finished_signal_.Wait(); returns control.

So basically those logs of yours are from different threads and mutex is unlock and lock by background_work_finished_signal_.Wait(); and your logs do not spot it.

So infact your logs should print something like this:

TEST_CompactMemTable mutex lock
TEST_CompactMemTable_wait_cv mutex unlock
BackgroundCallmutex lock
BackgroundCallmutex unlock
TEST_CompactMemTable_wait_cv mutex lock
TEST_CompactMemTable mutex unlock