linuxbashcurl

Log in to a site (Stack Overflow) by Bash


How can I log in to a website using Bash in Linux?

For example, to log in to Stack Overflow I tried many different methods, shown below, but nothing's worked.

wget --save-cookies cookies.txt --keep-session-cookies --post-data="username=blahblah&password=blahblahblha" "https://stackoverflow.com/users/login?ssrc=head&returnurl=https%3a%2f%2fstackoverflow.com%2f"

or

curl --user myusername:mypassword https://stackoverflow.com/users/login?ssrc=head&returnurl=https%3a%2f%2fstackoverflow.com%2f -v

I tried to inspect the element using Chrome to copy the curl request, but it didn't work (maybe it depends on a cookie and is only valid for a certain period of time).

Note that I need to login with username and password not with a cookie.


Solution

  • For fun only: Connection: keep-alive in bash.

    Introduction

    My goal was to build a Bash environment where the current console (or script) could deal interactively with an authenticated HTTPS session. This could be used to address lot of IoT targets, address any provider platform to monitor personal account, etc.

    Unfortunately, this could be badly used for stressing or hacking web platforms (targeting anyone) or even to targeting Stack Overflow's Fanatic badge (dirty cheater!).

    My apologies for any bad use of this. Anyway, I'm not responsible of what people do.

    Near pure Bash: require only OpenSSL:

    ... and gpg, but you're free to store your credential in another way.

    Preparing some values:

    #!/bin/bash
    shopt -s extglob
    
    URL='https://stackoverflow.com/'
    IFS=: read  -r user pass < <(gpg -qd <socred.gpg)
    
    IFS=/ read -r _ _ hst _ <<<"$URL"
    

    Running the OpenSSL executable as a background task:

    exec {wwwE}<> <(: -O)
    exec {wwwI}<> <(: -b)
    exec {wwwO}< <(exec stdbuf -o0  <&$wwwI 2>&$wwwE \
                       openssl s_client -quiet -connect "$hst":443 -ign_eof )
    osslpid=$!
    

    Then now, there is a little doReq function that will populate:

    :

    doReq() {
        hthead=() htbody=() hterr=()
        local target=$1 method=${2:-GET} head=true line cookies
        printf >&$wwwI '%s\r\n' "$method $target HTTP/1.1" "Host: $hst" \
               "User-Agent: aobs/0.01" "Connection: keep-alive" "Accept: */*"
        [ "$cookie" ] && printf >&$wwwI '%s' "$cookie"
        if [ "$method" = "POST" ];then
            printf >&$wwwI '%s\r\n%s\r\n\r\n%s' "Content-Length: ${#3}" \
                           'Content-Type: application/x-www-form-urlencoded' "$3"
        else printf >&$wwwI '\r\n'
        fi
        read -t 10 -ru $wwwO line
        htstatus=${line%$'\r'} ; hthead=("$htstatus")
        while read -t .3 -ru $wwwO line;do
            [ "${line%$'\r'}" ] || head=false;
            if $head ;then
                hthead+=("${line%$'\r'}");
                case $line in
                    [sS]et-[cC]ookie:* ) line=${line#*: };
                                     cookies+=("${line%%;*}");;
                esac
            else htbody+=("${line%$'\r'}") ;fi
        done
        if read -t 0 -ru $wwwE;then
            while read -t .1 -ru $wwwE line;do
                hterr+=("${line%$'\r'}")
                case $line in
                    depth* | verify* ) ;;
                    * ) echo "ERR: $line" ;;
        esac ; done ; fi
        [ ! -v "cookie" ] && [ "${cookies[0]}" ] &&
            printf -v cookie 'Cookie: %s\r\n' "${cookies[@]}"
    }
    

    Usage: doReq /file_part_of_URL [method] [post datas]

    Let's login:

    doReq /users/login POST "email=$user&password=$pass"
    

    Now show my badges:

    doReq /
    for ((i=${#htbody[@]};i--;)) ;do
        line="${htbody[i]}"
        case $line in
            *badge1* ) line="${htbody[i-1]}${htbody[i]}${htbody[i+1]}"
                       line=${line//>+([0-9])</><} line=${line//<*([^>])>}
                       printf '%b\n' "${line//&#9679;/ \\U25cf }" ;;
    esac ; done
    

    On my desk, with my account, this prints:

     ● 13 gold badges ● 88 silver badges ● 112 bronze badges
    

    ( at time this was posted! ;-).

    Of course, you're now ready to run doReq any time you like, as the connection stays open. We are now in environment/condition cited in the Introduction. (The version on my site does a forever loop, requesting this in a more efficient way, each rounded two minutes (EPOCHSECONDS % 120), up to user interaction. See at the bottom of this.)

    Playing in environment ...

    Trying a little function, avoid loop over $htbody to be quicker and

    showStat() {
      local line gold silver bronze item messages reputation
      read -r line <<< ${htbody[*]/#!(<span title=*)}
      mapfile -t line <<< "${line//\>/$'\n'}"
      for item in gold silver bronze; do
        printf -v $item ${line[*]/%!(* $item badges<\/span)};
      done
      messages=${htbody[*]/#!(role=*inbox messages*unread-count=*)};
      reputation=${htbody[*]/#!(<li class*your reputation: *)};
      printf -v out \
        "%b: %d, %b: %'d, %dx\360\237\237\241, %dx\342\232\252, %dx\360\237\237\244"\
          \\360\\237\\223\\$((252+( ${messages//[^0-9]}>2?2:${messages//[^0-9]})))\
          ${messages//[^0-9]} \\360\\237\\230\\216 ${reputation//[^0-9]}\
          $gold $silver $bronze
      printf '\e];  %s  \e\\%s\n' "$out"{,}
    }
    
    doReq /; showStat
    

    to show this stat in both standard output and console's title bar

    📪: 0, 😎: 71’617, 19x🟡, 131x⚪, 150x🟤
    

    Then:

    while ! read -sn1 -t 20;do doReq /; showStat;done
    

    Re-connect

    In case something went wrong, maybe you'd encounter a timeout. Before trying to reconnect, you have to forgot the cookie:

    unset cookie
    doReq /users/login POST "email=$user&password=$pass"
    doReq /; showStat
    

    or if subpid are gone ( ps $! don't show openssl ), you may have to re-run openssl s_client:

    unset cookie; exec {wwwO}<&-
    exec {wwwO}< <(exec stdbuf -o0  <&$wwwI 2>&$wwwE \
                   openssl s_client -quiet -connect "$hst":443 -ign_eof )
    doReq /users/login POST "email=$user&password=$pass"
    doReq /; showStat
    

    Exit

    Once you're done, before exit, you could stop openssl and close your file descriptor:

    (I've added ls and ps as debug commands.)

    ls -l /dev/fd/ ; ps --tty $(tty) ufw
    kill $osslpid
    exec {wwwE}<&-
    exec {wwwI}>&-
    exec {wwwO}<&-
    ls -l /dev/fd/ ; ps --tty $(tty) ufw
    

    This shows:

    total 0
    lrwx------ 1 user user 64 jui  2 13:52 0 -> /dev/pts/2
    l-wx------ 1 user user 64 jui  2 13:52 1 -> /dev/pts/2
    lrwx------ 1 user user 64 jui  2 13:52 10 -> pipe:[940266653]
    lrwx------ 1 user user 64 jui  2 13:52 11 -> pipe:[940266654]
    lr-x------ 1 user user 64 jui  2 13:52 12 -> pipe:[940266655]
    lrwx------ 1 user user 64 jui  2 13:52 2 -> /dev/pts/2
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    user    28110  0.0  0.0  11812  7144 pts/7    Ss   jui01   0:02 bash
    user    14038 30.0  0.0   9116  4228 pts/7    S+   13:52   0:00  \_ /bin/bash ./getSo.sh
    user    14045  0.5  0.0   9356  5856 pts/7    S+   13:52   0:00       \_ openssl s_client -quiet -connect stackoverflow.com:443
    user    14048  0.0  0.0  12404  3400 pts/7    R+   13:52   0:00       \_ ps --tty /dev/pts/7 ufw
    
    total 0
    lrwx------ 1 user user 64 jui  2 13:52 0 -> /dev/pts/2
    l-wx------ 1 user user 64 jui  2 13:52 1 -> /dev/pts/2
    lrwx------ 1 user user 64 jui  2 13:52 2 -> /dev/pts/2
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    user    28110  0.0  0.0  11812  7144 pts/7    Ss   jui01   0:02 bash
    user    14038 30.0  0.0   9632  4756 pts/7    S+   13:52   0:00  \_ /bin/bash ./getSo.sh
    user    14051  0.0  0.0  12404  3332 pts/7    R+   13:52   0:00      \_ ps --tty /dev/pts/7 ufw
    

    The OpenSSL process is done and all three file descriptors become closed.

    You could find this script (less condensed), with an extended main loop at getSo.sh.txt, getSo.sh.

    Bug (limitation)

    2022-09-29 Edit getSo.sh replacing badge1 by badge3. (Script work only if you already earnned at least one bronze badge!! This is a bug, but harmless than previous version which require a gold badge.)