There are a lot of tanky posts coming from lemmy.ml. Their whole purpose seems to be to troll and spread their bullshit far and wide. They are nearly as bad as the alt-right. They argue in bad faith and celebrate authoritarian oppression. The beehaw mods might want to consider defederating.
You should do what your users want. Growth in users for the sake of “line goes up” is a faulty attitude. Dozens of users sounds like a community of good faith users, very nice. You do you!