The cookie exchange initiated by HelloVerifyRequest described in section 4.2.1 of RFC 6347 is essential to DoS-protection in DTLS. Unless I'm overlooking something, the WolfSSL documentation does not really describe how to enable this cookie exchange with its DTLS implementation.
The closest I can find is the function wolfSSL_send_hrr_cookie, about which the manual states:
This function is called on the server side to indicate that a HelloRetryRequest message must contain a Cookie. The Cookie holds a hash of the current transcript so that another server process can handle the ClientHello in reply. The secret is used when generting the integrity check on the Cookie data.
As far as I understand, the HelloRetryRequest (compare HelloVerifyRequest) is TLS 1.3 terminology. DTLS 1.3 is not finalized. How does one enable and control the DTLS 1.0/1.2 cookie exchange in WolfSSL?
The DTLS Hello Cookie is enabled by default. The wolfSSL server code uses the callback function EmbedGenerateCookie() in the file src/wolfio.c to generate the cookie per the recommendation in RFC 6347 §4.2.1.
TLSv1.3's hrr_cookie is something else unrelated.