I'm trying to publish messages from the cloud function to the cloud Pub/Sub topic using their NodeJS client package. I'm facing the below error:
Total timeout of API google.pubsub.v1.The publisher exceeded 60000 milliseconds before any response was received.
Solutions I have tried:
Here is my code used to publish messages:
export class FeedService {
private static instance: FeedService;
private pubsubClient: PubSub;
private feedTopic: Topic;
private maxMessages = 50;
private maxMilliseconds = 10000;
private messagePromises: Promise<string>[] = [];
constructor() {
this.pubsubClient = new PubSub({
projectId: process.env.PROJECT_ID,
credentials: {
client_email: process.env.GOOGLE_CLOUD_CLIENT_EMAIL,
private_key: process.env.GOOGLE_CLOUD_CLIENT_PRIVATE_KEY,
},
});
}
public static getInstance() {
if (!FeedService.instance) {
FeedService.instance = new FeedService();
}
return FeedService.instance;
}
// provide global topic instance
private getFeedTopic() {
if (!this.feedTopic) {
this.feedTopic = this.pubsubClient.topic(process.env.FEED_TOPIC_ID, {
batching: {
maxMessages: this.maxMessages,
maxMilliseconds: this.maxMilliseconds,
},
});
}
return this.feedTopic;
}
// Calling this function to publish messages
async pushNotificationForAggregation(payload: FeedTypes.FeedQueuePayload) {
try {
const messageOptions: MessageOptions = {
json: payload,
publishTime: {
seconds: Date.now() / 1000,
},
};
// store promises
this.messagePromises.push(
this.getFeedTopic().publishMessage(messageOptions)
);
return { success: true };
} catch (error: any) {
logger.error(
`[Failed to push feed to queue] [${payload.userId}]: ${error?.message}`
);
return { success: false };
}
}
// await all promises before sending HTTP response
async processRemainingMessages() {
try {
const messages = await Promise.all(this.messagePromises);
return { success: true, messages };
} catch (error: any) {
logger.error(
`[Failed to process remaining messages] : ${error?.message}`
);
return { success: false };
}
}
}
Updates
I have tried the same code with the cloud run instead of the cloud function, and it works fine there.
So I can confirm there are no issues with the code.
I have solved this issue. The problem was with environmental variables. Google Cloud functions parse each variable added from the GUI console, and ads escape sequences. So \n
in my service account's private will be converted to \\n
resulting in wrong secrets.
It's strange that the Pub/Sub SDK does not return proper error messages; it always returns an API timeout, no matter how much time I increase the timeout.
Removing escape sequences from the account's private key solves the problem.