Sign in with Apple throws Apple Sign-In Error: [The operation couldn’t be completed (com.apple.AuthenticationServices.AuthorizationError error 1000.)]

I am trying to implement Signin with Apple using RNFirebase. I have exactly followed the the steps mentioned here but it is always giving me the following error

ERROR Apple Sign-In Error: [Error: The operation couldn’t be completed. (com.apple.AuthenticationServices.AuthorizationError error 1000.)]

I am testing using dev build (physical device) and also prod build using testflight and getting the same error.

I am making the builds using the following commands

eas build --profile development:device --platform ios

eas build --profile production --platform ios

You need to add the Capability in jour app.json/app.config.js if you use eas (https://docs.expo.dev/build-reference/ios-capabilities/ and https://docs.expo.dev/versions/latest/config/app/#entitlements).
So it should look something like this:

{
  "expo": {
    "name": "YourApp",
    "slug": "your-app",
    "scheme": "yourapp",
    "ios": {
      "bundleIdentifier": "com.yourcompany.yourapp",
      "buildNumber": "1.0.0",
      "supportsTablet": true,
      "entitlements": {
        "com.apple.developer.applesignin": ["Default"]
      }
    },
    "plugins": [
      "expo-apple-authentication"
    ]
  }
}