Can you be banned for nasty posting mean-spirited comments or 'hate speech' even if it has to do with the image? I don't see it in the ToS but I think I recently saw a whole comment section get purged.

If you have to be told "Don't be a dick" then you're not the kind of person who would follow rules in the first place.

Well there's actually rules about all that

See:
gelbooru.com/index.php?page=wiki&s=view&id=207

Rule n.98 of the secret Gelbooru Bible states: All hail Lozer and the mighty dev, our lords and saviors. Amen.

i regret to inform you that unwritten rules are how human social interactions work.

But this whole site has been spinning this narrative about "Not caving into pressure to remove art" but saying mean words in a comment section gets your account nuked without warning? I've seen way worse things said.
At that point it's less about "Social Interaction" and more about "The rules are whatever we want them to be at any given time." That's perfectly fine, it's their site. Just let me know beforehand so I can make a judgement call if I want to support the site with donations anymore.

do you remember that they have guidelines for said art, though

Posting art and posting comments are two separate subjects. If one doesn't like a particular kind of art, they can blacklist the tag(s) they dislike. Using the comment section to denounce or demean people has nothing to do with "not caving into pressure to remove art".

Not sure, but at least one of the mods (not Lozer himself) is into the SJW and cancel-culture type nonsense. I recall the admins saying actual banning is rare as IP bans affect every IP ever used by that user and thus their VPN, public networks, etc, but yes, removing comments and posts for certain offensive speech (read: not in line with western identity politics) is a pretty common thing. At least it was when I still had comments enabled, and I can only imagine it's gotten much worse to match our culture in the past few years. Mean spirited comments from "retard" all the way up to "kill yourself" over petty shit like different tastes in anime or fetishes was usually left untouched, though it could just be that most people were not sensitive enough to report those cases.

Either way, I don't think your account will be nuked unless you're spamming it or something. I'm sure I'll be corrected if I'm wrong.

If someone is posting comments the mods delete more than once, they'll mail them and tell them what to stop doing. If they keep doing it after that they'll get banned, and they can't say they didn't know any better at that point.