Recently, Google launched a new tool to build conversation agents using Gemini and Dialogflow. It is a new feature and I tried testing it out via the console where I am able to get it to work for some simple flows. However, the documentation regarding it is slightly confusing as right now it is supposedly a feature within Dialogflow. Going through its API documentation, I am unable to find a way to directly query the Agent API link. I also went through this detect intent API documentation, which was what we have previously used to interact with Dialogflow, but it does not state how to actually call this Agent API. I also tried seeing if just using the test console within Dialogflow would be enough but it seems the two are not directly integrated this way.
whereas on Agent console it works fine
As mentioned by @Anshuman Kumar in the comment, to resolve this issue follow this stack link.
Posting the answer as community wiki for the benefit of the community that might encounter this use case in the future.
Feel free to edit this answer for additional information.