Babepedia scraper

Plugins Scraper Development
1

Seems like alot of my scrapers seem to have broken. Not sure whats going on. Using truenas

Version

App Version:

v0.29.3

Version:

v1.0.16

025-12-12 01:30:43Error scrapeSinglePerformer: input: scrapeSinglePerformer error while name scraping with scraper Babepedia: scraper script error: exit status 69  
2025-12-12 01:30:43Error [Scrape / Babepedia] raise HTTPError(http_error_msg, response=self)  
2025-12-12 01:30:43Error [Scrape / Babepedia] File "/usr/lib/python3.12/site-packages/requests/models.py", line 1026, in raise_for_status  
2025-12-12 01:30:43Error [Scrape / Babepedia] scraped.raise_for_status()  
2025-12-12 01:30:43Error [Scrape / Babepedia] File "/root/.stash/scrapers/community/Babepedia/Babepedia.py", line 161, in performer_by_name  
2025-12-12 01:30:43Error [Scrape / Babepedia] ^^^^^^^^^^^^^^^^^^^^^^^  
2025-12-12 01:30:43Error [Scrape / Babepedia] result = performer_by_name(name)  
2025-12-12 01:30:43Error [Scrape / Babepedia] File "/root/.stash/scrapers/community/Babepedia/Babepedia.py", line 172, in <module>  
2025-12-12 01:30:43Error [Scrape / Babepedia] Traceback (most recent call last):  
2025-12-12 01:30:43Error [Scrape / Babepedia] requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.babepedia.com/ajax-search.php?term=Ai%20Sayama  
2025-12-12 01:30:43Error [Scrape / Babepedia] Running Python 3.12.12 (main, Oct 11 2025, 15:09:54) [GCC 14.2.0] at /usr/bin/python3  
2025-12-12 01:30:39Warning read transaction error on fetch performer image: reading from database: sql: Rows are closed  
2025-12-12 01:30:39Warning read transaction error on fetch performer image: reading from database: sql: Rows are closed
2

You are getting 403 forbidden and so am I. When I try the urls in the browser I get a Cloudlfare automation check. Once past that the actual urls work fine so it’s the Cloudflare protection causing the issue. Not sure there is much that can be done about that.

3

I believe a solution could likely be running the apps traffic through my desktop as an exit node utilizing wire guard. Then you can just change the IP if your scraper gets challenged. How to go about that within the truenas community edition space conveniently however, remains a challenge.

1 Like
4

Thinking about this more maybe I could vibe code(not a coder) to have something scrape locally then import using stashes api and graphql.

Would be more efficient in the sense that I dont have to worry about crazying natting or compromising the functionality of my other self hosted applications running on the same box

5

Each scraper is it’s own case. Babepedia upped their CF protection to a point where cloudscraper can’t get past. Unfortunately not much we can realistically do, IP reputation does play a part but it’s not that simple