After long struggle I managed to generate (using Ant) on MacOSX (Yosemite) a DMG (of modular JavaFX application that deploys to *.app folder) even with custom icons of the app and DMG volume (both stored as *.icns files in build's package/macosx folder). Drag and drop works, it installs the app, the app launches so everything is fine. Well, almost.
I was tempted to make also the custom background for DMG. It is placed together with the icons in package/macosx folder as App-background.png file (where App part is the name of my app, as it is with icons). And javapackager actually picks up the background, but unfortunately after opening DMG the background seems to be placed in windows' top-left corner, the window itself is like twice the size of the background, so 3/4 of the window's background is just white.
I tried with different PNG sizes, but to no avail. DMG window is always way bigger than the image.
Is there any way to make javapackager (aka fx:deploy which I actually use) to generate DMG with proper settings, that just make the DMG window fit background size?
Edit: It seems like DMG window has actually the size of the PNG (the original background file), but for some reason the image, when displayed as background, is "slightly" scaled down.
After long investigation I found the (not so obvious and rather not direct) reason for this misbehavior. Fortunately I found also the solution.
Apparently there's some problem with Finder handling PNG while displaying DMG file. It all comes down to chunk pHYs which contains 3 fields (4b, 4b and 1b long respectively). It is pixels per unit on x axis, pixels per unit on y axis and 0 or 1 for unit (0 no unit, 1 a meter unit). Playing with these values (via hex editor) did change somehow scaling behavior (and window size), but I wasn't able to figure out correct values. Eventually I get rid of this chunk completely (exporting from GIMP as PNG without saving resolution) and that solved the case.
So for anyone struggling with this: just make sure, there is no pHYs chunk in PNG file.
Edit: One can do that easily with ImageMagick:
$ convert image.png -define png:exclude-chunk=pHYs image.png
This strips pHYs chunk from the image.png file.