erlangyaws

yaws appmods Accept-Language A record is not correct

I work with yaws from a docker container for years, basically taken from https://github.com/segeda/docker-yaws/blob/master/Dockerfile:

FROM erlang:20-alpine
...
&& git clone https://github.com/klacke/yaws.git /yaws-src \
...

Things just used to run fine, but all of a sudden my code fails, and I cannot find the error(s). I could not even find a git version which worked when it had before, so I suspect it might not be a code fault, but then -- what could it be?

It cannot be my code, can it? Because I boiled it down to the example given in http://yaws.hyber.org/appmods.yaws and produced the same error:

%% this is my appmod called from yaws

-module(myurl).
-author('kklepper').
-include("../../include/yaws_api.hrl").
%-include("/usr/local/lib/yaws/include/yaws_api.hrl"). 
% relative or absolute -- either way same result
-export([out/1]).

-define(debug, true).
-ifdef(debug).
-define(trace(Str, X), io:format("Mod:~w line:~w ~p ~p~n", 
                                 [?MODULE,?LINE, Str, X])).
-else.
-define(trace(X, Y), true).
-endif.

box(Str) ->
    {'div',[{class,"box"}],
     {pre,[],Str}}.

out(A) ->
?trace('A', A),
    {ehtml,
     [{p,[],
       box(io_lib:format("A#arg.appmoddata = ~p~n"
                         "A#arg.appmod_prepath = ~p~n"
                         "A#arg.querydata = ~p~n",
                         [A#arg.appmoddata,
                          A#arg.appmod_prepath,
                          A#arg.querydata]))}]}.

The file yaws_api.hrl was and still is present:

/yaws # ls -la /usr/local/lib/yaws/include/yaws_api.hrl
-rw-r--r--    1 1000     50            5563 May 13  2018 /usr/local/lib/yaws/include/yaws_api.hrl

With trace you can see that the A record is not correct -- no A#arg.querydata etc. -- hence the error. Why?

Obviously, querydata, for example, as such is given: lg=en.

Mod:myurl line:22 'A' {arg,#Port<0.2934>,
                          {{10,255,0,2},52801},
                          {headers,"keep-alive",
                              "text/html, application/xhtml+xml, application/xml;q=0.9, image/webp, image/apng, */*;q=0.8, application/signed-exchange;v=b3",
                              "voxx.b2d",undefined,undefined,undefined,
                              undefined,undefined,undefined,undefined,
                              "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36",
                              undefined,[],undefined,undefined,undefined,
                              undefined,undefined,undefined,undefined,
                              undefined,
                              [{http_header,11,'Accept-Language',undefined,
                                   "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"},
                               {http_header,10,'Accept-Encoding',undefined,
                                   "gzip, deflate"},
                               {http_header,0,"Upgrade-Insecure-Requests",
                                   undefined,"1"}]},
                          {http_request,'GET',
                              {abs_path,"/industries?lg=en"},
                              {1,1}},
                          {http_request,'GET',
                              {abs_path,"/industries?lg=en"},
                              {1,1}},
                          undefined,"/industries","lg=en","industries","/ci",
                          "/","/ci/industries",undefined,undefined,<0.163.0>,
                          [],[],[],"/industries",myurl}

=ERROR REPORT==== 4-Oct-2019::00:46:06 ===


ERROR erlang code threw an uncaught exception:
 File: appmod:0
Class: error
Exception: {badrecord,arg}
Req: {http_request,'GET',{abs_path,"/industries?lg=en"},{1,1}}
Stack: [{myurl,out,1,
               [{file,"/usr/local/lib/yaws/voxx/ebin/myurl.erl"},{line,29}]},
        {yaws_server,deliver_dyn_part,8,
                     [{file,"yaws_server.erl"},{line,2921}]},
        {yaws_server,aloop,4,[{file,"yaws_server.erl"},{line,1274}]},
        {yaws_server,acceptor0,2,[{file,"yaws_server.erl"},{line,1073}]},
        {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]

By now I ran out of ideas. Any hint?

The data in question seems to be there. I am interested among others in language data and used to get the accepted language from A like so:

get_lang(A) -> 
    find_lang(find_http_header('Accept-Language', (A#arg.headers)#headers.other)).

find_http_header(Key,Headers) when is_list(Headers) ->
        case lists:keysearch(Key,3,Headers) of
            {value,{_,_,_,_,Value}} -> Value;
            false -> undefined
        end.

find_lang(AcceptLanguage) ->
    case AcceptLanguage of
        undefined ->
            "en";
        _ ->
            L = lists:nth(1,string:tokens(
                lists:nth(1,string:tokens(
                    lists:nth(1,string:tokens(AcceptLanguage, ";"))
                    ,","))
            ,"-")),
            L
    end.

Without the correct data structure, this cannot work.

Solution

  • A detail you included in your question shows that your appmod is getting a badrecord exception when trying to access the #arg record that Yaws passes to it. There are a couple ways to get such an exception:

    1. Pass an instance of a data type other than the expected record type to the function. This isn't happening in this case, since Yaws is calling the function and is correctly passing an #arg record instance.
    2. Compile the caller and the called function with two different definitions of the same record. We can simulate this problem in an Erlang shell as follows:
      • First, define a record #arg containing two fields
      • Define a function that takes an instance of that record as an argument and returns the value of one of the fields
      • Redefine the record in the shell to have just one field
      • Pass an instance of the redefined record to the function, which will cause a badrecord exception
            1> rd(arg, {f,g}).
        arg
        2> F = fun(A) -> A#arg.f end.
        #Fun<erl_eval.7.126501267>
        3> rd(arg, {f}).
        arg
        4> F(#arg{f=1}).
        ** exception error: {badrecord,arg}

    Since you're using Yaws 2.0.7 and you're also cloning Yaws from github, it's likely you're compiling some part of your code, probably your appmod, against the Yaws master branch and then running it against 2.0.7. Sometime after I tagged Yaws 2.0.7 on github, I accepted a change that added a new field to the #arg record definition. What this means is that any #arg record Yaws 2.0.7 creates and passes to your appmod will result in a badrecord exception since the appmod expects a record containing the additional field.

    I can think of a few ways to avoid this problem: