I'm writing a java code in which I run Spark batches
using Dataproc Serverless
. I have initially used Dataproc Client Library
for java and so far, it works just fine. However, by looking at Dataproc Serverless
documentation (https://cloud.google.com/dataproc-serverless/docs), there is not a single mention of interacting with it by using Java Client - it mentions REST, RPC and Cloud SDK.
I started wondering if I'm doing something wrong. Is using Dataproc Client Library
for Dataproc Serverless
related actions actually not recommended or somehow limited?
You're doing it right - the Dataproc Client Library is the correct library to interact with Dataproc Serverless (for context, it uses the gRPC API mentioned in the Dataproc Serverless documentation). Generally, Dataproc Serverless is a part of the overall Dataproc product. The fact that the Dataproc Client Library is not mentioned on the Dataproc Serverless docs page is a mistake.