javascriptnode.jspuppeteer

Puppeteer cant listen all request from paribu.com


I want to listen all request ( with data ) from paribu.com; but i listen only some request without request data. Etc. i see only this when i use sign ( and i dont unsterstand why i see same request two times ): https://www.paribu.com/auth/sign-in https://www.paribu.com/auth/sign-in Note :I see websocket received messages . I want to set alarm for "bitcoin" again i see only : "https://www.paribu.com/alarm".

The website is created entirely with JavaScript and sends data via websocket. But I couldn't figure out how the data is sent to the website. For example, when I want to log in, I can only see the paribu.com/auth/sign-in link on the log page. I can't see what else is being sent. I also want to make something like WhatsApp bot js. It will set automatic coin alarm and make automatic coin sales. Currently I can only process with incoming data.

var puppeteer = require("puppeteer");

async function run() {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();

  await page.goto("https://www.paribu.com");
  const cdp = await page.target().createCDPSession();
  await page.setViewport({width: 1080, height: 1024});
  await cdp.send("Network.enable");
  await cdp.send("Page.enable");

  const printResponse = function (response) {
    console.log("response: ", response);
  };

  cdp.on("Network.webSocketFrameReceived", printResponse); // Fired when WebSocket message is received.
  cdp.on("Network.webSocketFrameSent", printResponse); // Fired when WebSocket message is sent.

  page.on("request", request => {
    console.log(request.url());
  });

  page.on("response", response => {
    console.log(response.url());
  });
}

run();

Solution

  • The reason it doesn't work is because you are listening to the request after the page already loads and created websocket connections. If you start listening to the network before navigating to the page, it should work.

    websocket response in console

    const puppeteer = require("puppeteer");
    
    async function run() {
        // Launch the browser
        const browser = await puppeteer.launch({ headless: false });
        const page = await browser.newPage();
    
        // Create a CDP session to listen to network events
        const cdp = await page.target().createCDPSession();
        await cdp.send("Network.enable");
        await cdp.send("Page.enable");
    
        // Function to log WebSocket messages
        const logWebSocketMessage = (eventType, message) => {
            console.log(`${eventType}: `, message);
        };
    
        // Set up listeners for WebSocket frames
        cdp.on("Network.webSocketFrameReceived", (params) => logWebSocketMessage("Received", params));
        cdp.on("Network.webSocketFrameSent", (params) => logWebSocketMessage("Sent", params));
    
        // Set up listeners for HTTP requests and responses
        page.on("request", request => {
            console.log("Request URL: ", request.url());
        });
    
        page.on("response", response => {
            console.log("Response URL: ", response.url());
        });
    
        // Navigate to the target page
        await page.goto("https://www.paribu.com");
        
        // Set viewport size
        await page.setViewport({ width: 1080, height: 1024 });
    }
    
    // Execute the script
    run();
    

    To your other questions, you see two requests in console, because you wrote the code that way; you are printing both request and response, so it prints once for the outgoing request and another for the incoming response.

    You can access the data inside the request and response using various methods listed on their official doc. Here is the link to Request related api documentation.