NHentai scraping

BoneDude5 · October 31, 2025, 6:56am

I’ve installed your plugin but pressing any of the buttons doesn’t seem to change the string under the “Scraper User Agent” setting. I’ve tried putting my user-agent in there myself but no luck, I still get a “http error 403:Forbidden” error when I try to scrape.

How do I pass in my login cookies to Stash?

feederbox826 · October 31, 2025, 12:20pm

You can add cookies by modifying your scraper definitions

You can reference the DMM scraper for the syntax

github.com/stashapp/CommunityScrapers

scrapers/DMM.yml

6963885c3


      
          driver:
            cookies:
              - CookieURL: "https://www.dmm.co.jp"
                Cookies:
                  - Name: "age_check_done"
                    Domain: ".dmm.co.jp"
                    Value: "1"
                    Path: "/"
          # Last Updated Jan 10, 2024

BoneDude5 · November 1, 2025, 6:40am

Thank you, hopefully that can let me scrape NHentai.net again. They recently added Anubis and ever since then I haven’t been able to scrape, though the Anubis pop-up hasn’t been there in a week or two. I’ve been trying to find a way to get around it for a couple months.

feederbox826 · November 1, 2025, 6:21pm

with anubis, you need a completely different approach, in fact you’ll have much much better luck without a useragent set (or using the default golang UA)

github.com/TecharoHQ/anubis

docs/docs/design/how-anubis-works.mdx?plain=1

83a83e969


      
              Worker -- Successful challenge --> Main
              Main -- Validate challenge --> ValidateChallenge
              ValidateChallenge -- Return cookie --> Backend
              ValidateChallenge -- If anything is wrong --> Fail
          ```
          
          ## Challenge presentation
          
          Anubis decides to present a challenge using this logic:
          
          - User-Agent contains `"Mozilla"`
          - Request path is not in `/.well-known`, `/robots.txt`, or `/favicon.ico`
          - Request path is not obviously an RSS feed (ends with `.rss`, `.xml`, or `.atom`)
          
          This should ensure that git clients, RSS readers, and other low-harm clients can get through without issue, but high-risk clients such as browsers and AI scraper bots will get blocked.
          
          ```mermaid
          ---
          title: Challenge presentation logic
          ---

BoneDude5 · November 1, 2025, 8:58pm

Damn, so there’s no way to just solve the Anubis challenge on my browser and then have the scraper mimic me?
So what, will NHentai need a whole new scraper, because currently it’s just a YML file, but it sounds like it will need a Python script to solve the POW challenge from Anubis. That’s all I understand so far. I know some Python and YAML but I’ve never made something like this before

feederbox826 · November 2, 2025, 3:42am

No, anubis doesn’t challenge you if you don’t pass in a Mozilla User-Agent, PoW isn’t that difficult to break, you’ll probably just have issues with other scrapers instead

feederbox826 · November 2, 2025, 3:59am

NHentai is not using anubis it’s using CF. Please open up an issue where it’s appropiate GitHub - stashapp/CommunityScrapers: This is a public repository containing scrapers created by the Stash Community.

BoneDude5 · November 2, 2025, 8:08am

I swear it was using Anubis a week or two ago, it would give the splash screen (or loading screen, whatever it should be called) until very recently. I emailed NHentai about the issue, since they provide download links for the galleries I figured preventing normal users from scraping was not intended, maybe they listened to my email.

I submitted an issue when it first became a problem but I can’t find my issue when searching through them. I made a throwaway GitHub account because I didn’t want to connect my real one with a porn app, so I can’t login to find my submission. It’s just gone, so can’t trust that a new one will get addressed.

Unfortunately the current NHentai scraper doesn’t have a Python file, so I can’t just slap cloudscraper onto it and see if that helps.I’ll read through the other issues with the cloudflare label and your write-up on DDOS guards and see if I can put something together myself.