Most of the places on the internet say it stands for WEB INFormation.
I rather doubt it. The folder contains executables. Information is not a suitable name for it.
As far as I know, "INF" stands for "Information", as you said. It probably was named WEB-INF for similarity with the META-INF directory in JAR files. Sometimes the meaning of a directory changes so much over time that it no longer makes sense. For example, bin directories in Unix/Linux often contain non-binary "executable" files, such as shell scripts.