Exposing Neo4j Bolt using Kubernetes Ingress

I'm trying to build a Neo4j Learning Tool for some of our Trainings. I want to use Kubernetes to spin up a Neo4j Pod for each participant to use. Currently I struggle exposing the bolt endpoint using an Ingress and I don't know why. Here are my deployment configs:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: neo4j
  namespace: learn
  labels:
    app: neo-manager
    type: database
spec:
  replicas: 1
  selector:
    matchLabels:
      app: neo-manager
      type: database
  template:
    metadata:
      labels:
        app: neo-manager
        type: database
    spec:
      containers:
        - name: neo4j
          imagePullPolicy: IfNotPresent
          image: neo4j:3.5.6
          ports:
            - containerPort: 7474
            - containerPort: 7687
              protocol: TCP
---
kind: Service
apiVersion: v1
metadata:
  name: neo4j-service
  namespace: learn
  labels:
    app: neo-manager
    type: database
spec:
  selector:
    app: neo-manager
    type: database
  ports:
    - port: 7687
      targetPort: 7687
      name: bolt
      protocol: TCP
    - port: 7474
      targetPort: 7474
      name: client
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: neo4j-ingress
  namespace: learn
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - host: learn.neo4j.com
      http:
        paths:
          - path: /
            backend:
              serviceName: neo4j-service
              servicePort: 7474
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: learn
data:
  7687: "learn/neo4j-service:7687"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: learn
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-nginx
  template:
    metadata:
      labels:
        app: ingress-nginx
    spec:
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0-beta.16
          args:
            - /nginx-ingress-controller
            - --tcp-services-configmap=${POD_NAMESPACE}/tcp-services
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace

The client gets exposed nicely and it reachable under learn.neo4j.com but I don't know where to point it to to connect to the DB using bolt. Whatever I try, it fails saying ServiceUnavailable: Websocket Connection failure (WebSocket network error: The operation couldn’t be completed. Connection refused in the console). What am I missing?

It automatically tries to connect to port 7687 by default - if you enter the connection url http://learn.neo4j.bolt.com:80 (or https), it works.