2 Commits

Author	SHA1	Message	Date
Edward Loveall	6ccea391ed	Add a bunch of well-known, LLM scrapers to robots.txt ... Unknown if this will actually stop them, but at least I can show my intent. User agents sourced from https://darkvisitors.com/	2024-01-15 14:55:14 -05:00
Edward Loveall	fcf3eb14d0	Initial app	2021-05-01 17:03:38 -04:00