Baitbuddies.com Scraper Proposal

Plugins Scraper Development
1

Disclosure: following links have worked examples which lead to the studio’s website, which is gay porn.

I have a proposal for a baitbuddies.com community scraper (StashDB Studio link here), but I’d like to run the logic past here first.

Title: This studio doesn’t expose/have titles for scenes, so the title is built from the credited performers, in the order the studio’s own filename lists them. Worked example: Visiting Owen’s studio profile, the scenes previews have performers listed. The first scene in view is:

Release Date: 03/03/16
Bait: Owen
Straight: Scott DeMarco

This scene’s filename is BB448_M_Scott_Owen.MP4, crediting Scott before Owen, which translates into a title “Scott DeMarco and Owen”.

(Using “and” instead of “&” to maintain consistency with GEVI scrapes: Bait Buddies: Gay Erotic Video Index)

Accommodations have been made for Solo scenes, which would just use the performer’s credited name and no “&”.

Title (follow-up disambiguations): Some scenes are follow-ups with the same cast, which would otherwise scrape to an identical title. The site flags these inconsistently in the scene’s description, so the scraper checks the first couple of description paragraphs for a “Part N” marker and appends it to the title when found. Worked example: Beau has two scenes with the same performer, Marcus Rivers:

bb1012-m-beau-marcus-2066 description opens: “… Some of you might recognize this week’s bait guy…” no Part marker gets title “Beau and Marcus Rivers”

bb1013-m-beau-marcus-2072 description opens: “… Part 2 of Beau and Marcus!..” get title “Beau and Marcus Rivers, Part 2”

Studio Code: The site doesn’t expose a labelled scene code in the page, but the video’s internal filename (visible in the page <title> and og:title tags) is prefixed with the studio’s release code, e.g. BB448_M_Scott_Owen.MP4. Taking everything before the first underscore gives the code as the site itself names it.

Page title: BB448_M_Scott_Owen.MP4 - Videos - Bait Buddies
Studio Code: BB448

Details: Pulled from the “Description” tab on the scene page.

Image: Using https://baitbuddies.com/media/thumbs/custom_thumbs/{id}.jpg. That’s the image that previews on the player before pressing play on the scene.

Scene URL: .../video/bb448-m-scott-owen-mp4-436.html has id 436
Image: https://baitbuddies.com/media/thumbs/custom_thumbs/436.jpg

Date: No release date appears on the scene page. It only shows up on listing pages, in a performer’s profile page (/models/{slug}.html), as “Release Date” next to each of their scenes. Python rationale: Date is recovered with a second request: fetch the first credited performer’s model page, find the entry whose link matches this scene’s numeric ID, and read the adjacent Release Date.

  • https://www.baitbuddies.com/video/bb448-m-scott-owen-mp4-436.html has “Bait: Owen”
  • Owen’s profile has the same scene link with metadata: Release Date: 03/03/16
  • Date: 2016-03-03

Tags: The scene page exposes these, so using these as-is.

Performers: Taken from the same Bait:/Straight: credit block on the scene page as Title, each with a link to their model profile page (which is reused to look up Date).

Owen gets https://www.baitbuddies.com/models/owen-429.html
Scott DeMarco gets https://www.baitbuddies.com/models/scott-demarco-668.html

I’ve got this worked out locally, and tested against my own Stash instance.

2

Community decisions:

  • Title:
    • Is ordering performers by their order in the title/slug preferred?
    • Do we want “and” or “&” as the separator for this scraper?
      • Current behavior: “and” used.
      • Context: IAFD exclusively uses “and”, and GEVI mostly uses “and”.
    • Do we want “A, B, and C” or “A, B and C” when 3+ performers appear?
      • Current behavior: “A, B and C”
  • Studio Code:
    • Do we want codes capitalized or not? i.e. bb123 vs BB123
      • Current behaviour: bb123
  • Tags:
    • There is a current blacklist of the “Straight” tag here, because it doesn’t fit the StashDB definition of “Scenes with male and female scene partners, without action between male performers”. Should that be unblacklisted and removed by hand before submitting, or stay blacklisted?
  • Details/Description:
    • Currently, the scraper is scraping the description box “as-is”, when many of the scenes start with “Welcome back to BaitBuddies.com!”. Do we want that intro removed? Rationale: I see many submissions omitting this, and I don’t mind either way.
  • Extra metadata:
    • There is an option to automatically pull WAYBIG, GEVI, and IAFD links in (not affecting the metadata, simply adding the link source). Would that be preferred in this scraper?