htmlc++qtqnetworkaccessmanagerqnetworkreply

Get contents of website from QWebView using Qt 5.5.1


I'm using Qt 5.5.1 and I make a small browser using QWebview I want when
I open a web page and press a button, it get the content of the website like I use get method in QnetworkAccessManager without using it because I want to get the data from website that have a login page so the URL not change when I log in and have not post method to PHP to get the data.
for example when I log in to www.login.com the data of the login show on the same link
I need any idea I can solve this problem or if I can get the current data from website open in the QWebview
Note
When I login into the website and get the data from it by press on view source code in firefox the login data appear in the source code
This is what I tried

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow) {
        ui->setupUi(this);
        ui->webView->load(QUrl("https://www.login.com")); // load page that have user name and password field
        QWebPage *page = ui->webView->page(); // get the current page 
        manager = page->networkAccessManager(); // get the access manager from this page
    }

    MainWindow::~MainWindow() {
        delete ui;
    }
    // get button
    void MainWindow::on_pushButton_clicked() {
        reply = manager->get(QNetworkRequest(QUrl("https://www.login.com"))); // make get now after login
        connect(reply, SIGNAL(readyRead()),this,SLOT(readyRead()));
        connect(reply, SIGNAL(finished()),this, SLOT(finish()));
    }

    void MainWindow::readyRead() {
        QString str = QString::fromUtf8(reply->readAll()).trimmed(); // read the data
        ui->plainTextEdit->appendPlainText(str);
    }

but I get the data of the first page without the login. I want to get the page content after login, please give me any hint on what I should do.
update
I view the page source code from firefox get the text input name and use QUrlQuery to make post to it by the result was the first page without login this the part of HTML code I got the name of it

<label class="label-off" for="dwfrm_ordersignup_orderNo">Numéro de la commande</label> <input class="textinput required" id="anyname" type="text"  name="UserName"  value=""  maxlength="2147483647" placeholder="* UserName" data-missing-error="Saisis ton numéro de commande. "  data-parse-error="Ce contenu est invalide"  data-range-error="Ce contenu est trop long ou trop court"  required="required" />

and it has same code for the other field.
Code for I use in Qt to make a post

manager = new QNetworkAccessManager(this);
QUrlQuery query;
query.addQueryItem("UserName", "AAAA");
query.addQueryItem("Password", "BBB");
reply = manager->post(QNetworkRequest(QUrl(ui->lineEdit->text().trimmed())), query.toString().toUtf8());
connect(reply,&QNetworkReply::downloadProgress,this,&MainWindow::progress);
connect(reply,SIGNAL(readyRead()),this,SLOT(readyRead()));
connect(reply, SIGNAL(finished()), this,SLOT(finish()));

I try the post code with PHP page I made and it work the problem here it's only HTML page


Solution

  • I will use a different approach, instead to simulate a POST request, I will load the login page normally with QWebView and use Qt to fill the username, password and do a fake click on the submit button.

    On how to save the after login page how what you see, not the HTML code, a good way provided by Qt is render the web view to PDF, with a downside of lose the HTML code.

    If only the code is enough you can use webview->page()->mainFrame()->toHtml()

    See a simple example, please note that you need to adapt the code to your environment, analyze the login page, etc.

    void MainWindow::start()
    {
        connect(webview, &QWebView::loadFinished, this, &MainWindow::LogIn);
    
        QString html = "<html>"
                       "<body>"
                       "<form action=\"https://httpbin.org/post\" method=\"POST\">"
                       "Username:<br>"
                       "<input type=\"text\" name=\"usernameinput\" value=\"abc\">"
                       "<br>"
                       "Password:<br>"
                       "<input type=\"password\" name=\"passwordinput\" value=\"123\">"
                       "<br><br>"
                       "<button name=\"button1\">Submit</button>"
                       "</form>"
                       "</body>"
                       "</html>";
    
        webview->setHtml(html); //Load yours https://www.login.com, using setHtml just for example
    }
    
    void MainWindow::LogIn(bool ok)
    {
        disconnect(webview, &QWebView::loadFinished, this, &MainWindow::LogIn); //Disconnect the SIGNAL
    
        if (!ok)
            return;
    
        QWebElement document = webview->page()->mainFrame()->documentElement();
    
        QWebElement username = document.findFirst("input[name=usernameinput]"); //Find the first input with name=usernameinput
    
        if (username.isNull())
            return;
    
        username.setAttribute("value", "def"); //Change the value of the usernameinput input even
                                               //if it already has some value
    
        QWebElement password = document.findFirst("input[name=passwordinput]"); //Do the same for password
    
        if (password.isNull())
            return;
    
        password.setAttribute("value", "123456"); //Do the same for password
    
        QWebElement button = document.findFirst("button[name=button1]"); //Find the button with name "button1"
    
        if (button.isNull())
            return;
    
        connect(webview, &QWebView::loadFinished, this, &MainWindow::finished);
    
        button.evaluateJavaScript("this.click()"); //Do a fake click on the submit button
    }
    
    void MainWindow::finished(bool ok)
    {
        disconnect(webview, &QWebView::loadFinished, this, &MainWindow::finished);
    
        if (!ok)
            return;
    
        QByteArray data;
        QBuffer buffer(&data);
    
        if (!buffer.open(QIODevice::WriteOnly))
            return;
    
        QPdfWriter pdfwriter(&buffer);
    
        pdfwriter.setResolution(100); //In DPI
    
        webview->page()->setViewportSize(QSize(pdfwriter.width(), pdfwriter.height()));
    
        QPainter painter;
        painter.begin(&pdfwriter);
        webview->page()->mainFrame()->render(&painter);
        painter.end();
    
        buffer.close();
    
        qDebug() << "PDF Size:" << data.size(); //Now you have a PDF in memory stored on "data"
    }