Purpose of mipmaps for 2D sprites?

In current Unity,

For use in Unity.UI as conventional UI ..

for any "Sprite (2D and UI)", in fact it always defaults to having "Generate Mip Maps" turned ON. Every time you drop an image in, you have to turn that "off" and apply.

As noted in the comments, these days you can actually use world space UI canvasses, and indeed advanced users may indeed have (say) "buttons that float over the head of Zelda and they are in the far distance". However if you're a everyday Unity user adding a button, just turn it off :)

In Unity, "sprites" can still be positioned in 3D space. For example, on a world space canvas. Furthermore, mipmaps are used when the sprite is scaled. This is because the mipmap sampling is determined by the texel size rather than the distance.

If a sprite is flat and perfectly scaled then there is no reason to use mipmaps. This would likely apply to your icon example.

I suspect that it is enabled by default for 2D games where sprites will often not be perfectly scaled. To clarify, a sprite does not need to be on a canvas. Sprites can exist as their own GameObject with a Sprite Renderer (not on a canvas.) When this is the case, scaling the camera view will change the sprite's size on the screen resulting in mipmapping due to the texel size changing. This results in making the sprite always perfectly scaled challenging without a canvas.