Is there an equivalent to ASSERT_* for EXPECT_CALL, which ends the test on first uninteresting/unmatched call?
I am using gtest+gmock to test HW register programming sequences, so all EXPECT_CALL are under StrictMock and InSequence. If any call is missing or incorrect, every single one after that will likely fail due to misalignment. This generates a lot of noise in test output, making it difficult to find original failure.
Example issue:
#include <gmock/gmock.h>
#include <gtest/gtest.h>
#include <cstdint>
void hw_register_write(uint32_t, uint32_t) {}
class hw_register_access {
public:
virtual void write(uint32_t address, uint32_t value) = 0;
};
class hw_registers : public hw_register_access {
public:
void write(uint32_t address, uint32_t value) override {
return hw_register_write(address, value);
}
};
class mock_registers : public hw_register_access {
public:
MOCK_METHOD(void, write, (uint32_t address, uint32_t value), (override));
};
void program_foo(hw_register_access& reg) {
reg.write(0x10, 0x123);
reg.write(0x10, 0x456); // Bug, want to stop immediately
for (size_t i = 0; i < 10; i++) {
reg.write(0x20, 0x789); // Don't want noise from this
}
}
TEST(programming_sequence_test, program_foo) {
testing::StrictMock<mock_registers> mock{};
testing::InSequence seq{};
EXPECT_CALL(mock, write(0x10, 0x123));
EXPECT_CALL(mock, write(0x11, 0x456));
EXPECT_CALL(mock, write(0x20, 0x789)).Times(10);
program_foo(mock);
}
Because reg.write(0x20, 0x789) no longer matches the expectation due to unfulfilled prerequisites, this generates over 300 lines of errors, hiding the root issue:
Unexpected mock function call - returning directly.
Function call: write(16, 1110)
...
Expected arg #0: is equal to 17
Actual: 16
Is there an equivalent to
ASSERT_*forEXPECT_CALL, which ends the test
This is a misrepresentation of what the ASSERT_* macros do. An ASSERT_ macro does not end the test; it aborts the current function. Often, these seem like the same thing because, often, the current function is the test. However, that's not the case when your test calls a function that calls a function that calls.... Somehow you need the test to end from potentially deep inside a call stack.
You can signal that a test should end the same way you can signal an error condition from deep in a call stack – throw an exception. I know of two reasonable ways to set this up and still get useful output when the test fails: default action and default expectation. Below, I'll give some pros and cons of each approach, along with one way to do each.
Note: If your tests are compiled with exceptions disabled, it might be possible to run a "death test" instead, utilizing std::exit instead of throwing an exception. That's more complicated, though, so I will leave that to the interested reader as an exercise.
Throwing from a default action is more in line with how GoogleTest is designed. As such, you can choose to let the testing framework handle the exception (as a failure). Furthermore, the framework will provide all the feedback that it thinks might be useful for diagnosing the failure. One downside is that "all the feedback" consists of dozens of lines for the question's code. While this is better than hundreds, it still includes more information than might be necessary for an "in sequence" test, which leads to some sifting to find the relevant information. The other drawback I can think of is that this approach requires adding something to each EXPECT_CALL, which can get tiresome.
The basic idea is simple. After you define your mock, use ON_CALL to set the default action to throw an exception. For this demonstration, I will throw a string literal; you can, of course, change this to whatever works for you, as long as it is a copyable value.
testing::StrictMock<mock_registers> mock{};
ON_CALL(mock, write).WillByDefault(testing::Throw("Unexpected call"));
The complication is that your EXPECT_CALLs use the default action. So if you do nothing else, every call to write will throw, ending (and failing) your test. To counteract this, each EXPECT_CALL needs to be given an action to do (via WillOnce or WillRepeatedly). Simply returning is a fine action to take.
EXPECT_CALL(mock, write(0x10, 0x123)).WillOnce(testing::Return());
EXPECT_CALL(mock, write(0x11, 0x456)).WillOnce(testing::Return());
EXPECT_CALL(mock, write(0x20, 0x789)).Times(10).WillRepeatedly(testing::Return());
This can get tiresome and redundant. It might be better to write a macro to handle the repeated parts. Based on the question's code, one possibility is the following.
#define MY_EXPECT_CALL(Param1, Param2, NumTimes) \
EXPECT_CALL(mock, write(Param1, Param2)) \
.Times(NumTimes) \
.WillRepeatedly(testing::Return());
MY_EXPECT_CALL(0x10, 0x123, 1);
MY_EXPECT_CALL(0x11, 0x456, 1);
MY_EXPECT_CALL(0x20, 0x789, 10);
This gives you the desired limit of one unexpected mock function call. The output reports why this one call did not match any expectations, as well as which expectations were not satisfied, plus a failure for the exception being thrown. To diagnose this failure, you could – since your expectations are "in sequence" – skim down the "no match" reasons to the first not-saturated expectation and see why that did not match.
Expected arg #0: is equal to 17 Actual: 16
which tells you what went wrong. Not too onerous, I think, but still some work.
Throwing from a default expectation works against the framework to some extent, since you are expecting the unexpected. It risks confusing the person maintaining the code, and it falls on you to generate useful output. A benefit is that you can make the output more focused. Also, the extra setup is per-test rather than per-expectation, which might be attractive for tests with dozens of expectations.
The starting point is similar to the default action case, except EXPECT_CALL is used instead of ON_CALL.
testing::StrictMock<mock_registers> mock{};
// First draft
EXPECT_CALL(mock, write).WillRepeatedly(testing::Throw("Unexpected call"));
That one line nominally gives what was requested in that the test ends after the first unexpected call and passes if there are no unexpected calls. However, the output is now too brief. With just this, the test framework no longer reports why none of the other expectations matched. You do see which expectations were not satisfied, and the first of these should be the one with the parameter mismatch (because of InSequence), but you do not see what the parameters actually were.
To remedy this, you would need to supplement the output. This could be done by providing additional information in the thrown exception. There are various ways to do this; I'll keep things simple for this answer (you can make things more sophisticated as desired). Let's go with a custom exception type that stores the parameters to the unexpected function call.
struct my_exception : public std::runtime_error {
uint32_t param1;
uint32_t param2;
my_exception(const char* msg, uint32_t paramA, uint32_t paramB)
: std::runtime_error(msg), param1(paramA), param2(paramB) {}
};
Next, augment the default expectation to utilize this custom type.
testing::StrictMock<mock_registers> mock{};
EXPECT_CALL(mock, write).WillRepeatedly([](auto param1, auto param2) {
throw my_exception("Unexpected function call to write", param1, param2);
});
Finally, unlike the "default action" approach, do not rely on the testing framework to handle the exception. Catch it yourself so you can provide additional output. Be as verbose as you need to be. Here is a simple example to get you strated:
try {
program_foo(mock);
} catch (const my_exception& e) {
ADD_FAILURE() << e.what() << "(" << e.param1 << ", " << e.param2 << ")";
}
}
This gives
Failed Unexpected function call to write(16, 1110)
followed by the unsatisfied expectations. Since your expectations are "in sequence", you can compare the above line to the first unsatisfied expectation to see why it did not match. Unlike the "default action" approach, there is no need to skip over output; the relevant information is right there at the top of the failure report.
Choose whichever approach works better for you.