hw: Don't call visit_end_struct() after visit_start_struct() fails