I am trying to use boost::locale and std::stod together in a german locale where a comma is the decimal separator. Consider this code:
boost::locale::generator gen;
std::locale loc(""); // (1)
//std::locale loc = gen(""); // (2)
std::locale::global(loc);
std::cout.imbue(loc);
std::string s = "1,1"; //float string in german locale!
double d1 = std::stod(s);
std::cout << "d1: " << d1 << std::endl;
double d2 = 2.2;
std::cout << "d2: " << d2 << std::endl;
std::locale loc("") creates the correct locale and the output is
d1: 1,1
d2: 2,2
as I expect. When I comment out line (1) and uncomment line (2), the output is
d1: 1
d2: 2.2
The result for d2 is to be expected. As far as I understand boost::locale wants me to explicitly specify that d2 should be formated as a number and doing
std::cout << "d2: " << boost::locale::as::number << d2 << std::endl;
fixes the output to 2,2 again. The problem is that std::stod does not consider 1,1 as a valid floating point number anymore and truncates it to 1.
My question is: why does std::stod stops working when I generate my locale with boost::locale ?
Additional information: I am using VC++2015, Boost 1.60, no ICU, Windows 10
Update:
I noticed that the problem is fixed when I set the global locale twice, first with std::locale("") and then with boost:
std::locale::global(std::locale(""));
bl::generator gen;
std::locale::global(gen(""));
I have no idea why it behaves this way, though!
Long story short: boost::locale changes only the global c++-locale object, but not the C-locale. stod uses the C-locale and not the global c++-locale object. std::localechanges both: the global c++-locale object and the C locale.
The whole story: std::locale is a subtle thing and responsible for a lot of debugging!
Let's start with the c++ class std::locale:
std::locale loc("de_DE.utf8");
std::cout<<loc.name()<<"\n\n\n";
creates the German locale (if it is available on your machine, otherwise it throws), which results in de_DE.utf8 on the console.
However it does not change the global c++ locale object, which is created at the start-up of your program and is classical ("C"). The constructor of std::locale without arguments returns a copy of the global state:
...
std::locale loc2;
std::cout<<loc2.name()<<"\n\n\n";
Now you should see C if nothing messed up your locale before. std::locale("") would do some magic and find out the preferences of the user and return it as object, without changing the global state.
You can change the local state with std::local::global:
std::locale::global(loc);
std::locale loc3;
std::cout<<loc3.name()<<"\n\n\n";
The default constructor results this time in de_DE.utf8 on the console.
We can restore the global state to the classical by calling:
std::locale::global(std::locale::classic());
std::locale loc4;
std::cout<<loc4.name()<<"\n\n\n";
which should give you C again.
Now, when the std::cout is created it clones its locale from the global c++ state (here we do it with the stringstreams, but it the same). Later changes of the global state does not affect the stream:
//classical formating
std::stringstream c_stream;
//german formating:
std::locale::global(std::locale("de_DE.utf8"));
std::stringstream de_stream;
//same global locale, different results:
c_stream<<1.1;
de_stream<<1.1;
std::cout<<c_stream.str()<<" vs. "<<de_stream.str()<<"\n";
Gives you 1.1 vs. 1,1 - the first is the classical the second german
You can change the local locale-object of a stream with imbue(std::locale::classic()) it goes without saying, that this doesn't change the global state:
de_stream.imbue(std::locale::classic());
de_stream<<" vs. "<<1.1;
std::cout<<de_stream.str()<<"\n";
std::cout<<"global c++ state: "<<std::locale().name()<<"\n";
and you see:
1,1 vs. 1.1
global c++ state: de_DE.utf8
Now we are coming to std::stod. As you can imagine it uses the global c++ locale (not entirely true, bear with me) state and not the (private) state of the cout-stream:
std::cout<<std::stod("1.1")<<" vs. "<<std::stod("1,1")<<"\n";
gives you 1 vs. 1.1 because the global state is still "de_DE.utf8", so the first parsing stops at '.' but the local state of std::cout is still "C". After restoring the global state we get the classical behaviour:
std::locale::global(std::locale::classic());
std::cout<<std::stod("1.1")<<" vs. "<<std::stod("1,1")<<"\n";
Now the German "1,1" is not parsed properly: 1.1 vs. 1
Now you might think we are done, but there is more - I promised to tell you about std::stod.
Next to the global c++ locale there is so called (global) C locale (comes from the C language and not to be confused with the classical "C" locale). Each time we changed the global c++ locale the C locale has been changed too.
Getting/setting of the C locale can be done with std::setlocale(...). To query the current value run:
std::cout<<"(global) C locale is "<<std::setlocale(LC_ALL,NULL)<<"\n";
to see (global) C locale is C.To set the C locale run:
assert(std::setlocale(LC_ALL,"de_DE.utf8")!=NULL);
std::cout<<"(global) C locale is "<<std::setlocale(LC_ALL,NULL)<<"\n";
which yields (global) C locale is de_DE.utf8. But what is now the global c++ locale?
std::cout<<"global c++ state: "<<std::locale().name()<<"\n";
As you may expect, C knows nothing about c++ global locale and leaves it unchanged: global c++ state: C.
Now we are not in Kansas any more! The old c-functions would use the C-locale and new c++ function the global c++. Brace yourself for funny debugging!
What would you expect
std::cout<<"C: "<<std::stod("1.1")<<" vs. DE :"<<std::stod("1,1")<<"\n";
to do? std::stod is a brand-new c++11 function after all and it should use global c++ locale! Think again...:
1 vs. 1.1
It gets the German format right, because the C-locale is set to 'de_DE.utf8' and it uses old C-style functions under the hood.
Just for the sake of completeness, the std::streams use the global c++ locale:
std::stringstream stream;//creating with global c++ locale
stream<<1.1;
std::cout<<"I'm still in 'C' format: "<<stream.str()<<"\n";
gives you: I'm still in 'C' format: 1.1.
Edit: An alternative method to parse string without messing with global locale or be disturbed by it:
bool s2d(const std::string &str, double &val, const std::locale &loc=std::locale::classic()){
std::stringstream ss(str);
ss.imbue(loc);
ss>>val;
return ss.eof() && //all characters interpreted
!ss.fail(); //nothing went wrong
}
The following tests shows:
double d=0;
std::cout<<"1,1 parsed with German locale successfully :"<<s2d("1,1", d, std::locale("de_DE.utf8"))<<"\n";
std::cout<<"value retrieved: "<<d<<"\n\n";
d=0;
std::cout<<"1,1 parsed with Classical locale successfully :"<<s2d("1,1", d, std::locale::classic())<<"\n";
std::cout<<"value retrieved: "<<d<<"\n\n";
d=0;
std::cout<<"1.1 parsed with German locale successfully :"<<s2d("1.1", d, std::locale("de_DE.utf8"))<<"\n";
std::cout<<"value retrieved: "<<d<<"\n\n";
d=0;
std::cout<<"1.1 parsed with Classical locale successfully :"<<s2d("1.1", d, std::locale::classic())<<"\n";
std::cout<<"value retrieved: "<<d<<"\n\n";
That only the first and the last conversions are successful:
1,1 parsed with German locale successfully :1
value retrieved: 1.1
1,1 parsed with Classical locale successfully :0
value retrieved: 1
1.1 parsed with German locale successfully :0
value retrieved: 11
1.1 parsed with Classical locale successfully :1
value retrieved: 1.1
std::stringstream may be not the fastest but has its merits...