@jjg It seems you and whatever people are talking past each other. I don't think it's likely that the people you are talking to aren't understanding that people who are calling themselves antifa are anti fa.
More likely they are questioning the idea that punching nazis is effective and/or ethical.