Adding robots.txt support to FediDB, if you want to block FediDB from crawling your instance, please add a disallow rule to the ./well-known endpoint like GtS is doing.
https://goblin.technology/robots.txt
We will be dropping the Popular Accounts section as those are more harder to support, and I will re-enable them once I can get around to it.
@dansup I love fediDB.
@mike Glad to hear that!
I've been working on a big refactor that will be open sourced without the paid theme it's currently using, so others can run their own, and contribute to the main instance.