In the FEU settings, there's a field for "Image File Extensions that Users
allowed to upload:"
The check for these is case sensitive, but the field is fairly short so you
can't enter them all:
.gif,.png,.jpg,.jpeg,.GIF,.PNG,.JPG
(there's no room for .JPEG, let alone if you wanted to allow pdf, svg, or
something else that comes along)

TL;DR: please make the field longer, or don't make the file extension check case
sensitive.