Bounty to fix fc2ppvdb.yml scraper

fc2_lover · November 29, 2025, 2:07pm

hello, for couple months now the fc2ppvdb.yml scraper for fc2ppvdb.com (NSFW) does not work anymore because to load tags and performers it requires javascript.

I offer a 0.08 monero XMR cryptocurrency bounty (which is about 33$ at the time of posting) to anybody who manages to fix the scraper and posts it here and also if possible on github.

feederbox826 · November 29, 2025, 10:02pm

Here’s my attempt for it

hope it helps the next person!

fc2_lover · November 29, 2025, 11:08pm

thanks for the quick work. i get the error “scraper fc2ppvdb: parse error: expected string near offset 3942 of ‘partitionKey’“. not sure if i am just doing something wrong since im not really used to advanced scraper.

feederbox826 · November 29, 2025, 11:09pm

Yes, I couldn’t get it to work, CDP is still blocked. Hoping to pass on what i’ve learned to the next person to attempt it

feederbox826 · April 10, 2026, 12:26am

needs flaresolverr defined with environment variable FLARESOLVERR_URL - I use byparr. VPN networks blocked so that’s a common problem

fc2_lover · April 10, 2026, 11:32am

thank you! EDIT works now had to manually install playwright however it now just says the following in stash trace log

[Scrape / fc2ppvdb] Invalid JSON response from article info endpoint. Try again.

byparr log seems fine

INFO:     Started server process [2113804]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8191 (Press CTRL+C to quit)
INFO:     From: 127.0.0.1 at 2026-04-10: https://fc2ppvdb.com/articles/4566405
INFO:     Done https://fc2ppvdb.com/articles/4566405 in 5.47s
INFO:     127.0.0.1:51310 - "POST /v1 HTTP/1.1" 200 OK

feederbox826 · April 10, 2026, 2:48pm

You need to also populate the config fc2ppvdb and age_gate tokens otherwise it keeps redirecting you to login page

fc2_lover · April 10, 2026, 8:55pm

i do have the cookie data filled out in fc2ppvdb.py and config.ini gets auto generated with

F12 → Application → Cookies

fc2ppvdb_session =

[long cookie ID1]

age_pass =

[long cookie ID2]

i think its an issue with my setup

uv run main.py
Using version unknown
Log level set to INFO
INFO: Started server process [1482274]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8191 (Press CTRL+C to quit)
INFO: From: 127.0.0.1 at 2026-04-10: https://fc2ppvdb.com/articles/4566405
INFO: 127.0.0.1:57258 - “POST /v1 HTTP/1.1” 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/uvicorn/protocols/http/httptools_impl.py”, line 416, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
self.scope, self.receive, self.send
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/uvicorn/middleware/proxy_headers.py”, line 60, in call
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/fastapi/applications.py”, line 1160, in call
await super().call(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/applications.py”, line 107, in call
await self.middleware_stack(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/errors.py”, line 186, in call
raise exc
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/errors.py”, line 164, in call
await self.app(scope, receive, _send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/base.py”, line 191, in call
with recv_stream, send_stream, collapse_excgroups():
~~~~~~~~~~~~~~~~~~^^
File “/usr/lib/python3.14/contextlib.py”, line 162, in exit
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/_utils.py”, line 87, in collapse_excgroups
raise exc
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/base.py”, line 193, in call
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/user/Downloads/Byparr/src/middlewares.py”, line 22, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/base.py”, line 168, in call_next
raise app_exc from app_exc.cause or app_exc.context
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/base.py”, line 144, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/gzip.py”, line 29, in call
await responder(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/gzip.py”, line 130, in call
await super().call(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/gzip.py”, line 46, in call
await self.app(scope, receive, self.send_with_compression)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/middleware/exceptions.py”, line 63, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/_exception_handler.py”, line 53, in wrapped_app
raise exc
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/_exception_handler.py”, line 42, in wrapped_app
await app(scope, receive, sender)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/fastapi/middleware/asyncexitstack.py”, line 18, in call
await self.app(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/routing.py”, line 716, in call
await self.middleware_stack(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/routing.py”, line 736, in app
await route.handle(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/routing.py”, line 290, in handle
await self.app(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/fastapi/routing.py”, line 130, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/_exception_handler.py”, line 53, in wrapped_app
raise exc
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/starlette/_exception_handler.py”, line 42, in wrapped_app
await app(scope, receive, sender)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/fastapi/routing.py”, line 116, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/fastapi/routing.py”, line 670, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
…<3 lines>…
)
^
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/fastapi/routing.py”, line 324, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/user/Downloads/Byparr/src/endpoints.py”, line 68, in read_item
await dep.page.wait_for_load_state(
“networkidle”, timeout=timer.remaining() * 1000
)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/playwright/async_api/_generated.py”, line 9141, in wait_for_load_state
await self._impl_obj.wait_for_load_state(state=state, timeout=timeout)
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/playwright/_impl/_page.py”, line 572, in wait_for_load_state
return await self._main_frame.wait_for_load_state(**locals_to_params(locals()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/playwright/_impl/_frame.py”, line 273, in wait_for_load_state
return await self._wait_for_load_state_impl(state, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/user/Downloads/Byparr/.venv/lib/python3.14/site-packages/playwright/_impl/_frame.py”, line 301, in _wait_for_load_state_impl
await waiter.result()
playwright._impl._errors.TimeoutError: Timeout 49089.63791000133ms exceeded.

feederbox826 · April 10, 2026, 10:45pm

huh. I ran byparr in docker so can’t comment on that, it Just Worked for me

but you should populate in the ini instead of the py file. I’ll add an additiona log to check for the login page

fc2_lover · April 18, 2026, 1:20pm

thanks for your effort. i now use docker but still error don’t understand what i am doing wrong.

Debug Scraper script finished
Debug [Scrape / fc2ppvdb] age_pass cookie: longvalue2_%3D%3D
Error [Scrape / fc2ppvdb] fc2ppv_session ends with %3D, age_pass with %3D%3D
Error [Scrape / fc2ppvdb] Login prompt detected. Check cookies and try again.
Debug [Scrape / fc2ppvdb] fc2ppvdb_session cookie: longvalue1_%3D
Debug [Scrape / fc2ppvdb] cookies set, hitting article info endpoint
Debug [Scrape / fc2ppvdb] getting fresh cloudflare cookies

log into site
F12 to get fc2ppvdb_session & age_pass cookie values
put values into config.ini
fc2ppvdb_session = longvalue1_%3D
age_pass = longvalue2_%3D%3D

on http://localhost:8191/v1 i get “detail “Method Not Allowed”“ not sure if thats relevant

feederbox826 · April 18, 2026, 11:50pm

on http://localhost:8191/v1 i get “detail “Method Not Allowed”“ not sure if thats relevant

means its probably running

I’ll have another look, maybe it needs more cookies but I haven’t been able to reproduce anything just with those two cookies, maybe just put all cookies

feederbox826 · April 18, 2026, 11:51pm

and its not asn… still very confused why mine works just fine. is flaresolverr or stash going through a vpn or proxy?

feederbox826 · April 19, 2026, 12:08am

[fc2ppvdb] add xsrf-token and explicitly pass in cookies · stashapp/CommunityScrapers@0a9653f · GitHub added xsrf token, config will need to be manually updated

cornyburrito · April 24, 2026, 6:32pm

@feederbox826 I also got the “Login prompt detected” -error.

I removed the leading dots from the cookie domain definition and it started to work. I really don’t understand why, but rechecked without the alteration and got the login warning again. So that alteration really seemed to be the fix.

So for all three cookies:

domain=".fc2ppvdb.com"

to

domain="fc2ppvdb.com"

feederbox826 · April 24, 2026, 10:36pm

sure why not [fc2ppvdb] remove . from domains · stashapp/CommunityScrapers@b74569f · GitHub

fc2_lover · April 25, 2026, 11:39am

it works now! thank you very much you two

i only now have the issue that unless the filename only has the FC2-PPV id it errors out. for example filename like “hhd800.com@FC2-PPV-1234567”

Error scrapeSingleScene: input: scrapeSingleScene scraper fc2ppvdb: could not unmarshal json from script output: EOF

Error could not unmarshal json from script output: EOFError [Scrape / fc2ppvdb] Could not extract article ID from filename

the previous version of the scraper had some regex to ignore anything else in the filename.

queryURLReplace:
filename:

regex: .?(\d{5,}). #
with: $1

feederbox826 · April 25, 2026, 5:53pm

huh it should work the same and in my tests hhd800.com@FC2-PPV-1234567 regex does successfully extract. can you post a full filename that fails?

fc2_lover · April 25, 2026, 7:22pm

it doesn’t seem to matter what video id i use but for example “hhd800.com@FC2-PPV-4078398.mp4” always fails instantly while “4078398.mp4” works. it doesn’t even try to look it up just errors out.

ok it only works sometimes sometimes. idk why i renamed the video file bunch of times, re scanned and it always fails but now and then it works.

feederbox826 · April 26, 2026, 1:39am

the regex just captures the id and extracts it. Anything with 5 or more digits in a row is captured and queried.

cornyburrito · April 26, 2026, 12:52pm

I had this problem as well. You’ll need to use re.search() instead of re.match() which matches only at the beginning of the string. At least that change worked on my installation with the few files that I tested.