Looks like disabling anonymous comments didn’t fix it. Disabling anonymous shoutbox seems to have though. I really have no idea what’s causing this, but at least I can re-enable anonymous comments.