PyTorchMetrics Mean Absolute Percentage Error extremely high value

I am using PyTorch for a face recognition coursework and I have to calculate the MAPE value.

My first attempt was with torchmetrics.MeanAbsolutePercentageError class, but the result doesn't make sense.

For this reason I wrote a function to calculate it and it seems to work fine.

Investigating a bit, it seems to me the problem is related to the presence of 0 in the truth value array, but I didn't find anything in the torchmetrics documentation.

Is there a way to avoid this problem in torchmetrics?

Is it possible that the epsilon value in the MAPE formula is not set? If this is the case, how can I give it a value?

I am happy to use the other function, but I am curious to understand the reason of those results with torchmetrics.

These are the 2 function to calculate the MAPE:

def calculate_mape_torch(preds, targets):
    """Calculate MAPE using PyTorch method.
    
    Args:
        preds: array with ground truth values
        targets: array with predictions from model

    Returns:
        MAPE
    """
    if not isinstance(preds, torch.Tensor):
        preds = torch.tensor(preds)
    if not isinstance(targets, torch.Tensor):
        targets = torch.tensor(targets)

    mape = MeanAbsolutePercentageError()

    return mape(preds, targets) * 100


def calculate_mape(preds, targets, epsilon=1):
    """Calculate the Mean Absolute Percentage Error.
    
    Args:
        preds: array with ground truth values
        targets: array with predictions from model
        epsilon: value to avoid divide by zero problem

    Returns:
        MAPE
    """
    preds_flatten = preds.flatten("F")
    targets_flatten = targets.flatten("F")

    return np.sum(np.abs(targets_flatten - preds_flatten) / np.maximum(epsilon, targets_flatten)) / len(preds_flatten) * 100

With these values:

y_true = np.array([[1, 0, 3], [4, 5, 6]])
y_pred = np.array([[3, 2, 2], [7, 3, 6]])

the 2 functions give the results:

>>> calculate_mape(y_pred, y_true)
91.38888888888889

>>> calculate_mape_torch(y_pred, y_true)
tensor(28490084.)

With these values:

y_true = np.array([[1, 2, 3], [4, 5, 6]])
y_pred = np.array([[3, 2, 2], [7, 3, 6]])

the 2 functions give the results:

>>> calculate_mape(y_pred, y_true)
58.05555555555556

>>> calculate_mape_torch(y_pred, y_true)
tensor(58.0556)

They don't do a very good job of documenting, but their code uses an eps value of 1.17e-06. Using your calculate_mape with this eps gives the same result. This is just a result of eps scaling for zero value targets.

If your outputs are all int values, you might want to consider using a classification metric rather than a regression metric.