How to disable post-thumbnail from generation?

In my theme there is a set of

add_theme_support( 'post-thumbnails' );
set_post_thumbnail_size( 470, 680 );

which is actually generating one more image besides the default small,medium,medium-large,large

and it is set from the author of the theme for specific purposes of course.

I am using the wp_insert_attachment() default function for uploading image from front-end. From only this specific procedure i don’t want the generation of multiply images.Only the original image and the thumbnail image. So i made this simple code

function test_attachment( $user_data, $values, $user_id ) {

if( isset( $values['profile']['user_avatar']['path'] ) ) {

    $filename = $values['profile']['user_avatar']['path'];
    $filetype = wp_check_filetype( basename( $filename ), null );
    $wp_upload_dir = wp_upload_dir();
    $attachment = array(
        'guid'           => $wp_upload_dir['url'] . '/' . basename( $filename ),
        'post_mime_type' => $filetype['type'],
        'post_title'     => preg_replace( '/.[^.]+$/', '', basename( $filename ) ),
        'post_content'   => '',
        'post_status'    => 'inherit'
    );


    $attach_id = wp_insert_attachment( $attachment, $filename );


    require_once( ABSPATH . 'wp-admin/includes/image.php' );
    //remove all the generated images and just keep only the original and the thumbnail image.
    add_filter('intermediate_image_sizes_advanced', function($sizes) {
                    unset( $sizes['medium']);
                    unset( $sizes['medium_large']);
                    unset( $sizes['large']);
                    return $sizes;
        });


    $attach_data = wp_generate_attachment_metadata( $attach_id, $filename );
    wp_update_attachment_metadata( $attach_id, $attach_data );

    $previous_avatar_id = get_user_meta( $user_id, 'avatar_attachment_id', true );

    if( $previous_avatar_id ) {
        wp_delete_attachment( $previous_avatar_id );
    }

    update_user_meta( $user_id, 'avatar_attachment_id', $attach_id );

}
}  

which is working great and actually unsetting the generation of default image sizes with intermediate_image_sizes_advanced, but still there is a generation of the post-thumbnail image.
I have been trying to unset also the creation of the post-thumbnail image. I tried to add

unset( $sizes['post-thumbnail']);

but no luck.

How is possible to stop the generation of post-thumbnail image from inside the parser and without removing the original set_post_thumbnail_size as it is set there for other specific purposes?

Any ideas would be appreciated.