I just had an idea for keeping track of users who have been reported but we want to just keep an eye on them rather than blocking or silencing:
Have an account specifically for following problem-users, and give all mods access to it. Name it something that lets people know we're keeping an eye on them. If they take action against it, we can reciprocate.
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!