How to prevent users to view server files using WP File Manager plugin?

I’m developing some plugins to an website, but I don’t want the users to see the code (or download it). If the user install the plugin WP File Manager, all files in the host will be revealed to him. Plugins are an essencial part in the WP ecossystem and I don’t want to remove the ability of the users to install.

I just don’t want the users to access the core files.

I tried this approach:

1. Change the file owner, from www-data to ubuntu (or any server user).
2. Change the permissions, where directories uses 771 and files uses 661. In theses permissions, world can only execute the files;
3. Change the owner of plugins directory to www-data (for users to install plugins by admin).

I will update WP via wp-cli and the custom plugins via git.

With this change, the user can install new plugins normally. If the plugin WP File Manager are installed, no directories and no files are shown. All my custom plugins are invisible from WP File Manager, BUT, are invisible to WordPress too… The website and the admin load normally, only the plugins are not shown.

If I change the permission of my plugins to 775(dir) and 665(files), WP can load my plugins, but they are shown in WP File Manager too.

In the case in question, the user will not be the owner of the website, so he cannot have access to the files, but, it would be very bad to prevent users from installing plugins.

Other things I tried:

1. Use mu-plugins – same problem. And we have another problem, because activation hooks are not fired;
2. Disable file management PHP functions – these functions are used by WP core too;

How to prevent server files from being listed without preventing users from installing new plugins?