Scraping Region Geo specific data on sites

Plugins Scraper Development
1

Some sites have geo-location data, so there can be agreements over part of the metadata being scraped, as user x can see one thing vs user y in a different region.

Is there a plan to have some central scraping, so that a users setup does not cause differences it what StashDB wants to accept as a scrape?

Eg a central service that does the scrape or something within StashDB.

The alternative would also be to flag sites and urls automatically within the StashDB draft if a site is known to have geo-location data, different for some locations.

2

scrape-ci hopes to accomplish this Scrape-ci but implementation within stash is still very much up in the air. I’ve asked about potential plans or integrations, time will tell :slight_smile: There is a validator userscript and scraper floating around, but that is more on the user/ voter side

3

Thanks, I will write some code to scrape using that to get standard data, before I submit.

Or use Canada as the region and setup within a VPN