DupFileManager

David-Maisonave · May 11, 2025, 9:30am


	Summary	Manages duplicate files.
	Repository	https://github.com/stashapp/CommunityScripts/blob/main/plugins/DupFileManager
	Source URL	https://stashapp.github.io/CommunityScripts/stable/index.yml
	Install	How to install a plugin?

DupFileManager is a Stash plugin which manages duplicate files in the Stash system.
It has both task and tools-UI components.

Features

Creates a duplicate file report which can be accessed from the settings->tools menu options.The report is created as an HTML file and stored in local path under plugins\DupFileManager\report\DuplicateTagScenes.html.
- See screenshot at the bottom of this page for example report.
- Items on the left side of the report are the primary duplicates designated for deletion. By default, these duplicates are given a special _duplicate tag.
- Items on the right side of the report are designated as primary duplicates to keep. They usually have higher resolution, duration and/or preferred paths.
- The report has the following options:
  - Delete: Delete file and remove from Stash library.
  - Remove: Remove from Stash library.
  - Rename: Rename file.
  - Copy: Copy file from left (source) to right (to-keep).
  - Move: Copy file and metadata left to right.
  - Cpy-Name: Copy file name left to right.
  - Add-Exclude: Add exclude tag to scene,so that scene is excluded from deletion.
  - Remove-Tag: Remove duplicate tag from scene.
  - Flag-Scene: Flag (mark) scene in report as reviewed (or as requiring further review). Optional flags (yellow, green, orange, cyan, pink, red, strike-through, & disable-scene)
  - Merge: Copy Metadata (tags, performers,& studios) from left to right.
Can merge potential source in the duplicate file names for tag names, performers, and studios.
- Normally when Stash searches the file name for tag names, performers, and studios, it only does so using the primary file.
Advance menu (for specially tagged duplicates)

Screenshot 2024-11-22 1451391085×494 79.2 KB
- Advance menu can be access from the Settings->Tools->[DupFileManager Tools and Utilities] menu or from the reports.
  - Only access Advance Menu from the report when using Stash setup requiring a password.
- Here are some of the options available in the Advance Menu.
  - Delete only specially tagged duplicates in blacklist path.
  - Delete duplicates with specified file path.
  - Delete duplicates with specific string in File name.
  - Delete duplicates with specified file size range.
  - Delete with specified duration range.
  - Delete with resolution range.
  - Delete duplicates having specified tags.
  - Delete duplicates with specified rating.
  - Delete duplicates with any of the above combinations.
Bottom extended portion of the Advanced Menu screen.
- Screenshot 2024-11-22 232005952×311 35.5 KB
Delete duplicate file task with the following options:
- Tasks (Settings->Task->[Plugin Tasks]->DupFileManager)
  - Tag Duplicates - Set tag DuplicateMarkForDeletion to the duplicates with lower resolution, duration, file name length, and/or black list path.
  - Delete Tagged Duplicates - Delete scenes having DuplicateMarkForDeletion tag.
  - Delete Duplicates - Deletes duplicate files. Performs deletion without first tagging.
- Plugin UI options (Settings->Plugins->Plugins->[DupFileManager])
  - Has a 3 tier path selection to determine which duplicates to keep, and which should be candidates for deletions.
    - Whitelist - List of paths NOT to be deleted.
      - E.g. C:\Favorite,E:\MustKeep\
    - Gray-List - List of preferential paths to determine which duplicate should be the primary.
      - E.g. C:\2nd_Favorite,H:\ShouldKeep\
    - Blacklist - List of LEAST preferential paths to determine primary candidates for deletion.
      - E.g. C:\Downloads,F:\DeleteMeFirst\
  - Permanent Delete - Enable to permanently delete files, instead of moving files to trash can.
  - Max Dup Process - Use to limit the maximum files to process. Can be used to do a limited test run.
  - Merge Duplicate Tags - Before deletion, merge metadata from duplicate. E.g. Tag names, performers, studios, title, galleries, rating, details, etc…
  - Swap High Resolution - When enabled, swaps higher resolution files between whitelist and blacklist/graylist files.
  - Swap Longer Duration - When enabled, swaps scene with longer duration.
- Options available via DupFileManager_config.py
  - dup_path - Alternate path to move deleted files to. Example: “C:\TempDeleteFolder”
  - toRecycleBeforeSwap - When enabled, moves destination file to recycle bin before swapping files.
  - addPrimaryDupPathToDetails - If enabled, adds the primary duplicate path to the scene detail.
Tools UI Menu

Screenshot 2024-11-22 1455121002×587 28.2 KB
- Can access either Duplicate File Report (DupFileManager) or DupFileManager Tools and Utilities menu options.

Requirements

pip install --upgrade stashapp-tools
pip install requests
pip install Send2Trash

Installation

Follow Requirements instructions.
In the stash plugin directory (C:\Users\MyUserName.stash\plugins), create a folder named DupFileManager.
Copy all the plugin files to this folder.(C:\Users\MyUserName\.stash\plugins\DupFileManager).
Click the [Reload Plugins] button in Stash->Settings->Plugins->Plugins.

That’s it!!!

Options

Options are accessible in the GUI via Settings->Plugins->Plugins->[DupFileManager].
More options available in DupFileManager_config.py.

Screenshots

Example DupFileManager duplicate report. (file names have been edited to PG).
- The report displays preview videos that are playable. Will play a few seconds sample of the video. This requires scan setting [Generate animated image previews] to be enabled when scanning all files.
- Screenshot 2024-11-22 2253591717×1873 951 KB
- If there’s a scene on the left side that has a higher resolution or duration, it gets a yellow highlight on the report.
- There’s an optional setting that allows both preview videos and preview images to be displayed on the report. See settings htmlIncludeImagePreview in the DupFileManager_report_config.py file.
- There are many more options available for how the report is created. These options are targeted for more advanced users. The options are all available in the DupFileManager_report_config.py file, and the settings have commented descriptions preceeding them. See the DupFileManager_report_config.py file in the DupFileManager plugin folder for more details.
Tools UI Menu

Screenshot 2024-11-22 1455121002×587 28.2 KB
- Can access either Duplicate File Report (DupFileManager) or DupFileManager Tools and Utilities menu options.
DupFileManager Report Menu
- Screenshot 2024-11-22 151630652×367 22.9 KB
DupFileManager Tools and Utilities
- Screenshot 2024-11-22 152023509×640 89 KB
Full bottom extended portion of the Advanced Menu screen.
- Screenshot 2024-11-22 232208949×1114 136 KB

Future Planned Features

Add logic to merge performers and galaries seperatly from tag merging on report. Planned for 1.5.0 Version.
Add code to report to make it when the report updates the screen (due to tag merging), it stays in the same row position. Planned for 1.5.0 Version.
Add logic to merge group metadata when selecting merge option on report. Planned for 2.0.0 Version.
Add advanced menu directly to the Settings->Tools menu. Planned for 2.0.0 Version.
Add report directly to the Settings->Tools menu. Planned for 2.0.0 Version.

KlutzyBubbles · May 31, 2025, 4:21am

Has anyone been able to get this working? I am on windows and been trying to see if this can help me merge like 1000 duplicates but I can’t load the report even though the report file exists.

Does anyone know of any way to bulk merge duplicates? I have transcoded a bunch of files from mp4 to mkv but want to keep all the tags from the mp4s onto the mkv but the only way i have found to do this so far is one at a time in sceneDuplicateChecker

Hellfire702 · July 5, 2025, 3:24am

Don’t know if this is the right place to put this, but upon following the install instructions & attempting to run either tool from this plugin, it leads Stash to crash with the following error:

TypeError: AjaxData.responseJSON.data.runPluginOperation is null
    ToolsAndUtilities@http://localhost:9999/plugin/DupFileManager/javascript:244:9
    t@http://localhost:9999/assets/index-31134d58.js:40:9217
    apply@http://localhost:9999/assets/index-31134d58.js:44:185328
    t@http://localhost:9999/assets/index-31134d58.js:40:11169
    Suspense
    MY@http://localhost:9999/assets/index-31134d58.js:44:448755
    div
    jy2@http://localhost:9999/assets/index-31134d58.js:54:54934
    qg2@http://localhost:9999/assets/index-31134d58.js:49:4022
    C52@http://localhost:9999/assets/index-31134d58.js:44:188954
    Suspense
    M52@http://localhost:9999/assets/index-31134d58.js:44:186944
    Ug2@http://localhost:9999/assets/index-31134d58.js:49:3803
    apply@http://localhost:9999/assets/index-31134d58.js:44:185368
    TD2@http://localhost:9999/assets/index-31134d58.js:64:32661
    t@http://localhost:9999/assets/index-31134d58.js:40:75570
    MY@http://localhost:9999/assets/index-31134d58.js:44:448755
    c$2@http://localhost:9999/assets/index-31134d58.js:202:17903
    XZ@http://localhost:9999/assets/index-31134d58.js:19:183839
    t@http://localhost:9999/assets/index-31134d58.js:40:5733
    t@http://localhost:9999/assets/index-31134d58.js:40:12243

LudeDevil · July 18, 2025, 10:36pm

Hi there! DupFileManager looks like a fantastic addition to Stash. I am trying really hard to get it installed and working in my Docker image. I had to manually create the ‘reports’ directory, as the python code is looking in the wrong path. Once that was done, I get a new error (or warning) when running the report generation (No Tag). The error is:

[Plugin / DupFileManager] [LN:517] WRN: 'Exception calling [mergeMetadata]. Will retry; count(0); Error: \'id\'\nTraceBack=Traceback (most recent call last):\n  File "/config/plugins/community/DupFileManager/StashPluginHelper.py", line 521, in mergeMetadata\n    return self._mergeMetadata.merge(SrcData, DestData)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/config/plugins/community/DupFileManager/StashPluginHelper.py", line 996, in merge\n    self.mergeItems(\'performers\', \'performer_ids\', [])\n  File "/config/plugins/community/DupFileManager/StashPluginHelper.py", line 1045, in mergeItems\n    listToAdd += [item[\'id\']]\n                  ~~~~^^^^^^\nKeyError: \'id\'\n'

Does this give you any clue as to what is going wrong?

David-Maisonave · September 7, 2025, 9:17am

The author of this plugin does not monitor Discourse. Please post any bugs, feature request, help-request to the following link: https://github.com/David-Maisonave/Axter-Stash/issues/new/choose.
The Discourse link should be use for discussion that would be inappropriate in GitHub.

DogmaDragon · November 17, 2025, 9:19pm

rabidman

#General

I also tried "Dup file manager" . The tagging did nothing, also. No scene marked as DuplicateMarkForDeletion .

But my filter with "file count is greater than 1" shows 1346 files, which should have been tagged

rabidman

#General

the report generation seems to fail

25-11-15 05:46:34Error runPluginOperation: input: runPluginOperation exit status 1
2025-11-15 05:46:34Error [Plugin / DupFileManager] AttributeError: 'int' object has no attribute 'startswith'
2025-11-15 05:46:34Error [Plugin / DupFileManager] if stash.JSON_INPUT['args']['Target'].startswith("0"):
2025-11-15 05:46:34Error [Plugin / DupFileManager] File "/array/zfs/stash/plugins/DupFileManager/DupFileManager.py", line 155, in <module>
2025-11-15 05:46:34Error [Plugin / DupFileManager] Traceback (most recent call last):
2025-11-15 05:44:13Error [Plugin / DupFileManager] File "/array/zfs/stash/plugins/DupFileManager/DupFileManager.py", line 155, in <module>
2025-11-15 05:44:13Error runPluginOperation: input: runPluginOperation exit status 1
2025-11-15 05:44:13Error [Plugin / DupFileManager] Traceback (most recent call last):

curtiswtaylorjr · July 6, 2026, 1:30am

Several of the issues reported in this thread are documented with fixes in DupFileManager: 7 bugs affecting Docker/remote installs (report broken, dropdowns silent, external connection leakage) · Issue #35 · David-Maisonave/Axter-Stash · GitHub — covers the wrong report path (Docker install), file:// URL not loading in browser, and dropdown actions not working in Docker. Per the author’s note above, GitHub is where to track this.

curtiswtaylorjr · July 6, 2026, 1:32am

Just posted a response to KlutzyBubbles. Check the link to get help with your issue as it covers multiple bugs.

DogmaDragon · August 22, 2026, 1:59am

github.com/stashapp/CommunityScripts

DupFileManager, FileMonitor, RenameFile: bind the exception in retry handlers

main ← johnkattenhorn:fix/unbound-exception-variable

opened 07:41PM - 14 Aug 26 UTC

johnkattenhorn

+13 -13

### The problem Each retry loop in `StashPluginHelper.py` has two exception han…dlers: a narrow one for connection errors, and a broad one for everything else. Both build the same log message including `{e}` — but only the broad handler binds the name with `as e`. Because the sibling handler binds `e` as a function local, the narrow handler raises `UnboundLocalError` while formatting its own log message. That propagates out of the retry loop, so a dropped connection — the exact condition the loop exists to survive — aborts the operation on the first failure and reports the wrong error. Reproduction, matching the structure in the source: ```python def retry(): for i in range(3): try: raise ConnectionResetError("connection dropped") except (ConnectionResetError): print(f"retrying: {e}") # UnboundLocalError except Exception as e: print(f"retrying: {e}") ``` ### Scope 13 sites across the three plugins that vendor `StashPluginHelper.py`: | File | Lines | |---|---| | `plugins/DupFileManager/StashPluginHelper.py` | 522, 713, 748, 769 | | `plugins/FileMonitor/StashPluginHelper.py` | 522, 713, 748, 769 | | `plugins/RenameFile/StashPluginHelper.py` | 522, 713, 748, 769 | | `plugins/DupFileManager/DupFileManager.py` | 351 | Those are the `mergeMetadata`, `addTag`, `updateScene` and `destroyScene` retry loops in each copy of the helper, plus `setTagId_withRetry` in DupFileManager. Every change is the same one-token edit — adding `as e` to a handler that already interpolated `e`. No behaviour changes beyond the retry path now working as written. ### Testing Tested against a clean Stash **v0.31.1** instance in Docker with a two-scene library, driving the plugin helper through the real plugin entry path (plugin JSON on stdin, `server_connection` context). Fault injection: `mergeMetadata` was made to raise `ConnectionResetError` on its first two attempts, then succeed — the scenario the retry loop is written for. Same test run against both the current `main` and this branch: ``` RESULT[original]: UnboundLocalError: cannot access local variable 'e' where it is not associated with a value (attempts = 1) RESULT[patched]: survived the resets and completed; attempts = 3 ``` `main` aborts on the first reset. With the fix, the loop retries and completes as intended. Also confirmed: - A normal (non-failing) `mergeMetadata` between two scenes behaves identically before and after — no regression in the success path. - AST pass over the whole repository: zero remaining `except` handlers that reference an unbound `e`. - All four modified files compile. `plugins/starIdentifier/star_identifier.py` matches this pattern at first glance but is **not** affected — `load_encodings()` binds `e = Exception(...)` before the `try`, so its handler resolves the name from the enclosing scope. Deliberately left alone. ### Note for maintainers `StashPluginHelper.py` is vendored into three plugins and the copies have already drifted — DupFileManager's differs from the other two. This fix is applied to all three. Worth considering whether that file wants a single shared home. ### LLM-assisted contribution disclosure Per the repository's contribution policy: this change was prepared with LLM assistance (Claude). The diff has been reviewed by me, the testing described above was carried out and its results are reproduced verbatim, and I take full responsibility for the change and its license compliance.

@David-Maisonave would you be interested in reviewing it?

DogmaDragon · August 22, 2026, 2:02am

github.com/stashapp/CommunityScripts

DupFileManager: merge play and O history instead of discarding it

main ← johnkattenhorn:fix/merge-play-and-o-history

opened 07:41AM - 15 Aug 26 UTC

johnkattenhorn

+105 -3

Fixes #605. ### The problem `mergeMetadata.merge()` copies tags, performers, g…alleries, urls, studio, title, director, date, details, rating100 and code — but nothing about how the scene was watched. Play count, O count, total play duration and the `organized` flag are all left behind on the scene that is about to be deleted. So merge-then-delete silently destroys watch history. For a deduplication tool that is the one thing that must not happen: the file is replaceable, the history is not. ### Why the obvious fix does not work Setting `play_count` or `o_counter` through `update_scene` has no effect — both are `@deprecated(reason: "Unsupported")` on `SceneUpdateInput`, and the values are dropped server-side. `sceneAddPlay` and `sceneAddO` take a list of timestamps. That is a better fix than the issue asks for: the merged scene keeps *when* each play happened, not merely how many there were. ### What this does - **Play and O history** — replayed onto the destination with the source's original timestamps, via `sceneAddPlay` / `sceneAddO`. - **Play duration** — summed. Total time played is cumulative across two copies of the same content. - **`organized`** — follows the existing fill-the-blank rule used by every other scalar field: copied only when the destination does not already have it set. Counts and history are cumulative and therefore summed, not overwritten. Existing fill-the-blank behaviour for the other fields is untouched. ### Two fixes that only appeared under end-to-end testing Calling `mergeMetadata` directly passes full scene dictionaries, and against that the change worked first time. Running the plugin's real **Delete Duplicates** task told a different story, twice: 1. **`KeyError: 'organized'` — every merge failed.** `find_duplicate_scenes_diff` returns a reduced scene fragment, and `mergeItem` indexes fields directly. Merging `organized` raised on every pair, burned all five retries, and the merge never happened — while deletion carried on regardless. `mergeItem` now skips fields the query did not return. 2. **The history merge silently did nothing.** With the `KeyError` guarded, merges appeared to succeed, but that reduced fragment carries none of the playback fields, so every history field was skipped. `merge()` now re-reads both scenes when the playback fields are absent. Worth flagging for maintainers: when the merge fails, deletion still proceeds. That is how a reported merge failure becomes lost data rather than a skipped merge. ### Testing Against a clean Stash **v0.31.1** in Docker with two genuine phash duplicates (one source encoded at two qualities, distance 1), driven through the plugin's own **Delete Duplicates** task with **Merge Duplicate Tags** enabled. **Empty destination** — the doomed copy carries `play_count 4`, `o_counter 2`, `play_duration 250`, rating, title and `organized`: ``` surviving scene: master_hq.mp4 play_count=4 o_counter=2 play_duration=250 organized=True rating=95 play_history=['2026-08-01T10:00:00Z','2026-08-02T11:00:00Z','2026-08-03T12:00:00Z','2026-08-04T13:00:00Z'] o_history=['2026-08-05T14:00:00Z','2026-08-06T15:00:00Z'] ``` **Destination with its own history** — the case where both copies were watched: ``` before: low(delete) plays 4, Os 2, duration 250 | hq(keep) plays 1, O 1, duration 50 after : play_count 5 (expect 5) o_counter 3 (expect 3) play_duration 300 (expect 300) history lists are the union of both; title "HQ Keeps Its Title" preserved ``` Also covered: the integer-id entry point used by the report path (`mergeMetadata(int, int)`) — counts and duration sum correctly there too. Before this change, the same fixture produced `play_count 4 -> 0`, `o_counter 2 -> 0`, `organized true -> false` on the surviving scene. ### Scope Applied to all three copies of `StashPluginHelper.py`, which are byte-identical in this region. FileMonitor and RenameFile are covered by a compile check only — neither appears to exercise the merge class. Deliberately left for separate changes, to keep this reviewable: - `stash_ids` (StashDB links) and scene markers are still not merged. - The `# ToDo: Figure out how to merge groups` at the top of `merge()` is untouched. - `resume_time` is intentionally *not* merged — unlike the others it is a playback position rather than an accumulated total, and there is no obviously correct answer when both scenes have one. Independent of #757 — different region of the same file, so the two apply in either order. ### LLM-assisted contribution disclosure Per the repository's contribution policy: this change was prepared with LLM assistance (Claude). The diff has been reviewed by me, the testing described above was carried out and its results are reproduced verbatim, and I take full responsibility for the change and its license compliance.

@David-Maisonave would you be interested in reviewing it?