While trying to use Shrinksafe custom_rhino.jar to build Dojo I get MalformedInputException. The problem occurs when build reaches custom widgets/templates which contain french letters stored in UTF-8. The AIX machine has LANG=en_US which should be correct, judging by other documented problems regarding MalformedInputException with IBM JRE.
Switching to Sun's JRE is not acceptable solution as this build must run on IBM AIX. It is possible that a solution might be in changing something in AIX or a setting in IBM JRE or both. So far I've been unsuccessful.
Problem is also described in dojo forum but without proper resolution.
In the linked forum, I didn't see a clarification about the default character encoding on your build machine.
It may be that Dojo is using an encoding of UTF-8, but in fact your files are encoded with something like ISO-8859-1 (I'm assuming western Latin characters are used for French).
Do you have an editor such as Eclipse's that allows you to specify the character encoding to use on a particular file? You could try to open the file with UTF-8 encoding and see if the characters are what you expect.