[SOLVED] Can't get Python GCP Cloud Run gRPC service to respond

Can't get Python GCP Cloud Run gRPC service to respond

I am developing a chat app that has three main components (so far). I shall described the "architecture" of the working local development environment:

a client in SolidJS running in vite, using nice-grpc-web (a little TypeScript gRPC-web library)
an Envoy proxy running in a Docker container
a Python gRPC server that streams responses from Haystack's PromptNode, running directly on my machine but also tested and happy to run in a Docker container

This is all working fine locally 👍

With a view to moving it into the cloud, I deployed the Python app in a Docker container as a Cloud Run service on GCP.

Now, this service is running happily and is looging that it is happy and is responding to health checks. So I think that is all good.

However, reconfiguring my Envoy proxy to point to this service, rather than my local Python gRPC server, causes the whole thing to break. I get messages from Envoy telling me that the endpoints can't be reached:

ClientError: /ChatBot/AskQuestion UNAVAILABLE: upstream connect error or disconnect/reset before headers. reset reason: connection failure

So it is either the Envoy config that is making it unhappy, or the Cloud Run service that is making it unhappy. I have done lots of reading around both, but can't work it out.

Things that I imagine might be causing it:

the protocol is different. Now it is running over https, might I need to add more config to Envoy to support this?
I had to override the port on Cloud Run (standard is 8080, I am using 50051). Although it is responding on this port to the health checks from GCP, is there some further config I need to do?
perhaps a CORS issue? But I would expect the response to reflect that
although I have checked to make sure it is available to the internet and not just internally, could this be causing an issue?

The only even vaguely helpful article I have come across is this one, but that doesn't work either.

Here is my envoy.yaml, actual service name elided:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address: { address: 0.0.0.0, port_value: 8080 }
      filter_chains:
        - filters:
          - name: envoy.filters.network.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              codec_type: auto
              stat_prefix: ingress_http
              route_config:
                name: local_route
                virtual_hosts:
                  - name: local_service
                    domains: ["*"]
                    routes:
                      - match: { prefix: "/" }
                        route:
                          cluster: chat_service
                          timeout: 0s
                          max_stream_duration:
                            grpc_timeout_header_max: 0s
                    cors:
                      allow_origin_string_match:
                        - prefix: "*"
                      allow_methods: GET, PUT, DELETE, POST, OPTIONS
                      allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
                      max_age: "1728000"
                      expose_headers: custom-header-1,grpc-status,grpc-message
              http_filters:
                - name: envoy.filters.http.grpc_web
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_web.v3.GrpcWeb
                - name: envoy.filters.http.cors
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors
                - name: envoy.filters.http.router
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
    - name: chat_service
      type: logical_dns
      dns_lookup_family: V4_ONLY
      connect_timeout: 20s
      http2_protocol_options: {}
      lb_policy: round_robin
      typed_extension_protocol_options:
        envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
          "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
          explicit_http_config:
            http2_protocol_options: { }
      load_assignment:
        cluster_name: cluster_0
        endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: myapp.a.run.app
                    port_value: 50051

Dockerfile

FROM envoyproxy/envoy:v1.26.1

COPY envoy.yaml /etc/envoy/envoy.yaml

EXPOSE 9901
EXPOSE 50051
EXPOSE 50052
EXPOSE 8080

ENTRYPOINT [ "/usr/local/bin/envoy" ]
CMD [ "-c /etc/envoy/envoy.yaml", "-l trace", "--log-path /tmp/envoy_info.log" ]

Suggestions please!

Solution

Your envoy proxy needs to connect to your Cloud Run URL on port 443. There's a few layers of proxies in between your app and the Internet, so the port your application runs on isn't exposed publicly.