I'm building an application using Scryer Prolog that has a HTTP Client component.
It would be nice to streaming DCG to parse the data but phrase_from_stream/2 does not terminate when using the instantiated stream from http_open/3.
I appreciate debugging tips (how/where it doesn't terminate?).
Steps to reproduce in top-level scryer-prolog:
(Started a simple local HTTP server using python -m http.server 8000).
?- use_module(library(charsio)).
true.
?- use_module(library(dcgs)).
true.
?- use_module(library(http/http_open)).
true.
% reading the entire stream before parsing works as expected:
?- http_open("http://127.0.0.1:8000", Stream, []), get_n_chars(Stream, N, Chars), phrase(seq(Data), Chars).
Stream = '$stream'(0x600000636908), N = 726, Chars = "<!DOCTYPE HTML>\n<ht ...", Data = "<!DOCTYPE HTML>\n<ht ...".
?- use_module(library(pio)).
true.
% phrase_from_stream/2 does not terminate in this case:
?- http_open("http://127.0.0.1:8000", Stream, []), phrase_from_stream(seq(Data), Stream).
% nontermination
This seems to happen with other grammar rules as well (I'm originally trying to parse JSON data), but I'm able to reproduce with any rule. I also attempted to unify with content size and set Connection: close header (using options) without success.
Similar code using SWI-Prolog works as expected:
?- use_module(library(http/http_open)).
true.
?- use_module(library(dcg/basics)).
true.
?- use_module(library(pio)).
true.
?- http_open("http://127.0.0.1:8000", Stream, []), phrase_from_stream(string(Data), Stream).
Stream = <stream>(0x600002954400,0x600002974b00),
Data = [60, 33, 68, 79, 67, 84, 89, 80, 69|...] ;
false.
This was a bug in Scryer Prolog, as the streaming code was incomplete, giving always the response that there were more incoming bytes even it wasn't the case.