CommunityScrapers master branch breaking change

Development
1

As dicussed previously in Discord 1 2

We now have a breaking candidate and decision to make with v0.29. Because of stash#5793 (Group scrapers are entirely broken), “Group” type scrapers do not work with any release version below the upcoming v0.29. CommunityScrapers#2229 was made to transition it under the impression that v0.27 deprecated movieByURL and groupByURL was functional. CommunityScrapers#2368 will be a breaking change as any group scrapers will not work below v0.29

What is the best course of action here?

  • maintain two seperate branches to now actually match develop and stable
  • continue with the breaking migration
    • remove pseudo stable branch in favor of develop to avoid building twice and halve the build times
  • maintain a laststable branch and exclude breaking commits
  • abandon/ delay the migration for the next release
  • backport the scraper fix into a 0.28 bugfix to maintain compatibility
2

I think removing stable branch is the worst option as https://stashapp.github.io/CommunityScripts/stable/index.yml is pre-configured source index for everyone. It would require config file migration to change.

@Maista, @WithoutPants your thoughts?

3

well we can remove the double build and just copy the files over, that shouldn’t be a big deal. The issue is the references to it both in the builder and as a branch that has gone unused

4

This is my preference and original intention. stable was always intended as compatible with the last release, with develop being aligned with the develop branch on the stash repo.

What’s not clear is what process to follow for maintaining the two branches.

In general, non-breaking changes which are compatible with the current stable release should be committed to stable, which should be regularly merged into the develop branch. Changes that are compatible only with the develop branch should be committed to the develop branch of the CommunityScrapers/CommunityScripts repos, with the develop branches merged into stable after release.

If the current state of the stable branch is broken for 0.28, then we either need to reverse the migration on the stable branch and leave it on the develop branch instead, or I’ll need to do another bugfix release, which is probably not a terrible idea in any case.

5

As far as I can remember, there were only two breaking changes

  • MovieByURL → GroupByURL (upcoming)
  • The addition of Image/Gallery scraping
    • this would trigger a validation error, similar to groupByURL index out of range

As for other updates,

  • Selector/ compatibility updates should be pushed to both
  • Schema/ stashapp-tools updates are usually limited to CommunityScrapers as we use py_common

I’m not sure there is enough difference to warrant keeping (and maintaining) both branches since this would still break instances behind the bugfix version. In practice, we would end up with a desync between users on develop and stable, where issues minor reported won’t be available until the next stable release, or until a dev cherrypicks fixes over just to avoid a specific commit

Instead, I would like to propose switching the error of groupByURL index out of range into a WARN similar to Unknown scrape type that would allow for future forwards-compatibility with any and all new scraper types, while backwards compatibility will follow the same path. This, combined with the bugfix should cover almost all cases for forwards compatibility and avoid the hard fork at v0.29 and below (but still inevitably break any versions before bugfix)

2 Likes