If WP_DEBUG is not set, as I understand it, you should never ever see warnings. But on some sites on some servers, I’m still seeing a few. Not all the warnings that would be displayed if WP_DEBUG was set, but a select few.
I’ve tried changing the error level in php.ini, but that seems to have no effect on whether warnings appear or not, but they do appear in differing amounts on different servers (i.e. no warnings on development, one warning on staging, and a few more warnings on production).