How to add class to parent a tag with a sub menu

I have been trying to add a class to the parent a tag of a sub menu using a custom walker. The current walker I am using adds the class to the parent <li>, does anybody know how to adjust this to add the class to the parent <a> tag instead.

Here is the existing walker I am using:

class My_Walker_Nav_Menu extends Walker_Nav_Menu{
  public function display_element($el, &$children, $max_depth, $depth = 0, $args, &$output){
    $id = $this->db_fields['id'];    

    if(isset($children[$el->$id]))
      $el->classes[] = 'toggle-sub-nav closed';    

    parent::display_element($el, $children, $max_depth, $depth, $args, $output);
  }
}

Here is the code that is output:

Here is the code that I am aiming for: