Opensips - Bad From Header when I was using failover in my script

my REQUEST_ROUTE:

...
route [FROM_GATEWAY] {
    if ($fU=~"^999" && !$avp(new_caller)) {
        for ($var(v) in $(json(foo)[*]))
            $avp(new_caller) = $var(v);
    }

    if (!$avp(new_caller)) {
        $avp(new_caller) = $fU;
    }

    if (!do_routing($avp(new_caller),,,$avp(dst_rule_attrs),$avp(dst_gw_attrs), ,$avp(partition))) {...}

    $var(new_caller_domain) = ...;
    $var(uri) := "sip:" + $avp(new_caller) + "@" + $var(new_caller_domain);
    uac_replace_from("", "$var(uri)");


    $var(new_callee) = ...;
    $var(uri) := "sip:" + $var(new_callee) + "@" + $(ru{uri.domain});
    uac_replace_to("", "$var(uri)");

    route(RELAY);
}

my FAILURE_ROUTE:

...
failure_route [MANAGE_FAILURE] {
    if (t_check_status("403")) {
        $avp(new_caller) = NULL;
        if ($avp(new_caller)) {
            $var(isFailOver) = 1;
            route(FROM_GATEWAY);
        }
    }
...
}   

Invite to host1 is normal, but when inviting to host 2, there is a syntax error in the 'from' and 'to' fields:

From: <sip:1111111111@42.96.x.x><sip:2222222222@42.96.x.x>;tag=XDDpmaS4Fr8vK

To: <sip:3333333333@125.212.y.y><sip:3333333333@103.63.z.z>

Can someone explain what happened during when retry invite to another host?

Indeed! Although your sequence leads to OpenSIPS producing a malformed SIP header, given the current implementation of lazy SIP header mangling, this is expected. Long story short: you should never call uac_replace_from() and uac_replace_to() more than one time during the processing of a single SIP message (i.e. before it gets relayed).

But why does this happen during failure route? Well, it's because the SIP message gets saved together with all un-applied mangling instructions, such that when you re-try the routing inside failure route, you are in a very similar state to the one before route(RELAY)! You then re-run the processing route, together with a 2nd set of uac_replace_from() and produce a malformed SIP packet.

Can I mitigate this? Yes, you can! In order to avoid saving the mangling instructions, try to perform them inside a branch route instead! By doing so, the original message to be re-routed gets saved cleanly, without any lazy (un-applied) SIP header changes. Some code snippets:

route
{

    ...
    t_on_branch("NORMALIZE_HEADERS");
    route(RELAY);
}

branch_route [NORMALIZE_HEADERS]
{
    $var(new_caller_domain) = ...;
    $var(uri) := "sip:" + $avp(new_caller) + "@" + $var(new_caller_domain);
    uac_replace_from("", "$var(uri)");

    $var(new_callee) = ...;
    $var(uri) := "sip:" + $var(new_callee) + "@" + $(ru{uri.domain});
    uac_replace_to("", "$var(uri)");
}