The threat of abuse is a barrier to attending sports events for more disabled fans than ever, according to a new report.