pythonray

Is it necessary to use object store in python ray.serve?


In python's ray.core, you can use an object store that can be shared by each Actor as follows.

import numpy as np
import ray

@ray.remote
def Actor(a):
    return

a = np.zeros((5000, 5000))
result_ids = [Actor.remote(a) for x in range(10)]
results = ray.get(result_ids)

Is there any way to use object store in the same way in ray.serve if I have a large object like np.zeros((5000, 5000)) ? (or do I need to use it?)

I tried the same description method as ray.core. However, self.obj remained as object id and could not be referenced as a remote.

import ray
from ray import serve
from fastapi import FastAPI
import numpy as np

fast_app = FastAPI()

a = np.zeros((5000, 5000))
aid = ray.put(a)

@serve.deployment()
@serve.ingress(fast_app)
class Actor:
    def __init__(self,obj):
        self.obj = obj # <- not obj

    @fast_app.get("/obj")
    def health_check(self):
        return "test"

app = Actor.bind(aid) 

Solution

  • why not

    @serve.deployment()
    @serve.ingress(fast_app)
    class Actor:
        def __init__(self):
            self.obj = ray.put(np.zeros((5000, 5000)))
    
        @fast_app.get("/obj")
        def health_check(self):
            return {"obj": ray.get(self.obj)}