Skip to content

Releases: jaimetur/PhotoMigrator

v4.0.0

28 Mar 07:04
@jaimetur jaimetur
v4.0.0
a2d8e7c

Choose a tag to compare

View all tags
v4.0.0 Latest
Latest

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases

Release: v4.0.0

Release Date: 2026-03-20

🚨 Breaking Changes:

🌟 New Features:

  • Added full NextCloud Photos integration (WebDAV-based) across CLI, execution modes, automatic migration, and web interface (Issue: #567).
  • Added a dedicated NextCloud Photos tab in the web interface, placed between Immich Photos and Other Features (Issue: #567).
  • Added ClassNextCloudPhotos backend service with album/assets upload, download, cleanup, and rename/remove workflows.
  • Added Google Photos integration (official Library API based) across CLI, execution modes, automatic migration, and web interface.
  • Added a dedicated Google Photos tab in the web interface (next to Google Takeout).
  • Added Administration Panel for admin users to create/edit/delete users and configure per-user subpaths for /app/data and /app/volumes.
  • Added per-user Config.ini persistence in SQLite database, encrypting sensitive values at rest.
  • Added structured Configuration File tab editor based on sections/fields (Google Takeout, Google Photos, Synology Photos, Immich Photos, NextCloud Photos, TimeZone) with per-field help extracted from config comments.
  • Added a new Upload to Server tab in the web interface with destination-folder picker, separate Upload Local Folder and Upload Local Zip actions, and optional ZIP extraction mode (Extract ZIPs on upload: Yes/No).
  • Added ClassGooglePhotos backend service with OAuth refresh-token auth and supported upload/download modules.
  • Added secure multi-user mode in Docker Web Interface with login/session authentication and bootstrap admin credentials (admin / admin123 by default).
  • Added a dedicated Background Progress panel in Automatic Migration Live Dashboard to render Local Folder analysis progress (folder scan + files scan in current folder) without polluting the logs panel. (Issue #1037)

🚀 Enhancements:

  • Extended automatic migration endpoint parsing in web UI to support nextcloud[-photos][-1..3] for both source and target.
  • Added Google Photos OAuth credentials support in Config.ini and config loader ([Google Photos] section with account 1/2/3).
  • Extended automatic migration endpoint parsing in web UI to support google[-photos][-1..3] for both source and target.
  • Updated CLI/source-target validation and help text to include nextcloud and google-photos as cloud clients.
  • Improved name-pattern handling across cloud modules: wildcard-only patterns like * are now accepted for album selection, and literal patterns with special regex characters (for example album names containing parentheses) now work for matching/replacing without requiring manual escaping.
  • Improved album rename matching for literal album names containing regex metacharacters (such as parentheses), so rename workflows work without manual escaping.
  • Improved markdown render to detect italic font and bold+italic.
  • Improved album download progress visualization for cloud clients (NextCloud, Synology, Immich, Google Photos) by showing a nested per-album assets progress bar labeled as Downloading '<AlbumName>' Assets.
  • Improved album upload progress visualization for cloud clients (NextCloud, Synology, Immich, Google Photos) by showing a nested per-album assets progress bar labeled as Uploading '<AlbumName>' Assets.
  • Restricted folder/file browser API access to each user's assigned subfolders only, and enforced the same path restrictions at command execution time.
  • Enhancements in Web UI.
  • Forbidden Import/Export/Save configuration file in demo roles.
  • Added NextCloud dual-folder configuration in Config.ini and config loader with per-account NEXTCLOUD_PHOTOS_FOLDER_<id> (assets/no-albums) and NEXTCLOUD_ALBUMS_FOLDER_<id> (folder-based albums).
  • Separated markdown render in a new static file
  • Smoothed Automatic Migration Live Dashboard rendering by switching to manual refresh-on-change updates (dirty panels only), stabilizing Background Progress row order, and reducing completion-row churn to minimize panel flicker/tremor in CLI.
  • Added vertical scroll to the Access Logs table in the Administration Panel, limiting visible height (about 20 recent entries) while keeping older entries accessible by scrolling.
  • Improved Background Progress label normalization to keep only text after :, trim trailing : variants (:, :, :), and remove trailing path clauses such as in ... / in folder ... (Linux/Windows paths, quoted or unquoted).
  • Improved Background Progress title normalization to keep only the text after : (when present) and then trim surrounding whitespace.
  • Improved CLI Live Dashboard Background Progress routing for Takeout post-processing tqdm lines by accepting additional prefixes (including TQDM ...) and indeterminate progress frames (for example 81 files [00:00, ...]) so those updates are rendered in the Background Progress panel instead of the logs panel.
  • Improved CLI Live Dashboard responsiveness on terminal resize by syncing layout dimensions on each refresh cycle and recalculating visible Logs Panel rows from the current panel height.
  • Refined CLI Live Dashboard log coloring rules to style only explicit Automatic Migration events (Asset Pulled, Asset Pushed, Asset Duplicated, Album Created, Album Pulled, Album Pushed, Asset Fail/Failed) instead of broad keyword matches.
  • Updated GPTH to version 5.0.6 which includes several enhancements and bug fixing.

🐛 Bug fixes:

  • Fixed web command/help text normalization so --client=nextcloud examples are parsed consistently in UI descriptions.
  • Fixed Automatic Migration cloud-session initialization for NextCloud and Google Photos clients by enabling lazy thread-safe auto-login on first API call, preventing session is not initialized. Call login() first errors in source/target worker flows.
  • Fixed Automatic Migration dashboard crash in frozen binaries when Rich unicode tables are missing (ModuleNotFoundError: rich._unicode_data.unicode17-0-0) by adding packaging includes for rich._unicode_data and graceful dashboard fallback.
  • Fixed NextCloud Download All/Download Assets path behavior to scan NEXTCLOUD_PHOTOS_FOLDER_<id> recursively while excluding NEXTCLOUD_ALBUMS_FOLDER_<id> when nested, preventing duplicate album downloads and supporting direct /Photos layouts.
  • Other bug fixing.

📚 Documentation:

  • Added NextCloud Photos help page and linked it from README/help index.
  • Added Google Photos help page and linked it from README/help index.
  • Updated CLI/configuration/automatic-migration docs to include NextCloud and Google Photos support.
  • Changed header styles in help documents.
  • Updated documentation with all changes.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

Full Changelog: v3.8.0...v4.0.0

Assets 9
Loading
0 Join discussion

v3.8.0

20 Mar 19:37
@jaimetur jaimetur
v3.8.0
ba7d2a5

Choose a tag to compare

View all tags
v3.8.0

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases

Release: v3.8.0

Release Date: 2026-03-17

🚨 Breaking Changes:

🌟 New Features:

  • New Web Interface to configure and execute the different features & modules directly.
  • Added themes support to Web Interface.
  • Docker support to deploy the tool in docker and expose the new Web Interface (default port: 6078).
  • New docker-compose.yml and .env file to easily configure and deploy the tool with docker compose.
  • Added deterministic Live Photo upload support for Immich by pairing photo+video companions (same basename) and linking them through Immich API (livePhotoVideoId) across Immich upload features and Automatic Migration.
  • Added automatic burst stacking for Immich uploads (including Automatic Migration) using conservative heuristics (same folder, normalized basename, short time window, and size-ratio guard), creating Immich stacks with preferred primary asset ordering.

🚀 Enhancements:

  • Web Live Dashboard on Automatic Migration feature.
  • Adjusted Goggle Photos Panel layout.
  • Enhancements in Execution Log windows to parse progress bar properly.
  • Allow page reload while task is running.
  • Enhancements in Web interface layout and feature descriptions.
  • Added confirmation dialog on those modules that remove/rename/merge assets/albums on Synology/Immich Photos features.
  • Added confirmation dialog on Auto Rename Folders Content Based feature.
  • Improved Immich Remove Orphan Assets module to detect unsupported newer Immich API versions and abort gracefully with a clear message instead of failing with a raw 404 error.
  • Added a safeguard in organize_files_by_date to skip files already organized in the expected date folder structure (year, year/month, year-month) and avoid redundant re-nesting.
  • Improved Web Interface execution log buffering for long-running jobs with progress bars: progress refresh updates no longer flood line history, the in-memory log is compacted by progress key, and default web log buffer size was increased and made configurable via PHOTOMIGRATOR_WEB_MAX_JOB_OUTPUT_LINES.
  • Simplified docker-compose.yml file.
  • Horizontal scroll on log panel when lines are too large.
  • Improved markdown render for code blocks.

🐛 Bug fixes:

  • Added validation for --google-takeout path to block reserved special-folder names (Archive, Trash, Locked folder) and abort early with a clear message; same validation is enforced in Web UI folder selection (Issue #1008).
  • Fixed a queue accounting bug in Automatic Migration that could stall execution when processing album folders still marked as active (.active) in parallel workers (Issue #1009).
  • Fixed Automatic Migration startup with --log-level=DEBUG by passing profiling label arguments correctly to profile_and_print (Issue #1009).
  • Fixed Google Takeout GPTH input root normalization to handle direct Google Photos paths reliably (without copy fallback), preventing "Discover Media = 0" scenarios on NAS layouts (Issue #1013).
  • Fixed LocalFolder.get_albums_owned_by_user() raising UnboundLocalError (albums_filtered) when called with filter_assets=False during automatic migration album checks (Issue #1014).
  • Fixed Synology Live Photo downloads where ZIP payloads could be saved with .HEIF extension; ZIP payloads are now detected and extracted properly (Issue #1028).
  • Fixed download/migration EXIF date overwrite behavior: EXIF date tags are now only filled when missing, preserving existing shooting dates (Issue #1029).
  • Fixed Exiftool command line overflowing max length (#1052).
  • Fixed Exiftool not embebed on docker-dev version.
  • Fixed Download All (Immich/Synology) creating redundant ALL_PHOTOS/YYYY/MM/YYYY/MM folders by avoiding a duplicate date-organization pass after assets are already downloaded directly into YYYY/MM.
  • Fixed Immich album deletion status handling to treat HTTP 204 No Content (and any 2xx) as success, avoiding false warnings like Failed to remove album ... Status: 204.
  • Fixed Immich Live Photo linking reliability during uploads by retrying transient 404 responses when setting livePhotoVideoId, and skipping link attempts when either media component was uploaded as duplicate.
  • Fixed Web Interface log color classification to prioritize the explicit line log level prefix (e.g. WARNING:), preventing warning lines from being painted as errors when message text contains words like Client Error.
  • Fixed Web Interface active-job reconnect endpoint routing conflict by prioritizing /api/jobs/_active over dynamic /api/jobs/{job_id} matching, preventing false {"detail":"Job not found"} responses when querying active jobs.
  • Other bug fixing.

📚 Documentation:

  • Included Documentation links on Web interface.
  • Made link relatives to current release.
  • Updated documentation with all changes.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

What's Changed

New Contributors

Full Changelog: v3.6.2...v3.8.0

Contributors

stefan-sherwood, zhijiee, and jaimetur
Loading
0 Join discussion

v3.7.1

16 Mar 23:22
@jaimetur jaimetur
v3.7.1
3183033

Choose a tag to compare

View all tags
v3.7.1

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases+

Release: v3.7.1

Release Date: 2026-03-16

🚨 Breaking Changes:

🌟 New Features:

🚀 Enhancements:

🐛 Bug fixes:

  • Fixed stream Immich multipart upload to avoid OOM.

📚 Documentation:

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

What's Changed

New Contributors

Full Changelog: v3.6.2...v3.7.1

Contributors

zhijiee and jaimetur
Loading
0 Join discussion

v3.7.0

09 Jan 17:41
@jaimetur jaimetur
v3.7.0
2f19c06

Choose a tag to compare

View all tags
v3.7.0

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases+

Release: v3.7.0

  • Release Date: 2026-01-09

  • Main Changes:

    • 🚨 Breaking Changes:

    • 🌟 New Features:

    • 🚀 Enhancements:

      • Most of the Code Translated into English.

    • 🐛 Bug fixes:

    • 📚 Documentation:

      • Most of the Comments Translated into English.
      • Updated documentation with all changes.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

What's Changed

New Contributors

Full Changelog: v3.6.1...v3.7.0

Contributors

WizardOfYendor1 and jaimetur
Loading
0 Join discussion

v3.6.2

09 Jan 17:40
@jaimetur jaimetur
v3.6.2
2f19c06

Choose a tag to compare

View all tags
v3.6.2

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases+

Release: v3.6.2

  • Release Date: 2025-09-11

  • Main Changes:

    • 🚨 Breaking Changes:

      • Now your special folders (Archive, Trash, Locked folder) will be directly moved to Special Folders subfolder within the output directory in Google Takeout Processing feature.

    • 🌟 New Features:

      • Added support for Special Folders management such as Archive, Trash, Locked folder during Google Takeout Processing feature.

    • 🐛 Bug fixes:

      • Fixed bug in guess dates from filepath where it was looking into the grandparents folder even if the parent folder was not a month folder.
      • Fixed bug in guess dates from filename where it was detecting false positives in filenames as hashnames (i.e.: "5752a2c4-1908-4696-9117-fdfa750fbd88.jpg" --> incorrect 1908).
      • Fixed bug when extract zips that contains non-supporting folders for special system such as Synology NAS or SMB mounted devices such as folders ending with Whitespace that was renamed by the system as *_ADMIN_Whitespace_conflict*. Now native extractor makes an intelligent sanitization of folder names before to extract the archive.

    • 📚 Documentation:

      • Updated documentation with all changes.

    • 🚀 Enhancements:

      • Added new function to Sanitize (fix folders/files ending with spaces or dosts to avoid SMB mingling names) your Takeout input folder during Google Takeout Procesing Feature.

      • Improved Folder Analyzer to read date tags from groups EXIF/XMP/QuickTime/Track/Media/Matroska/RIFF only (other groups are not reliable).

      • Improved Folder Analyzer to choose the oldest date with time when two or more candidates has the same date but one has time o other not.

      • Improved Auto Rename Albums feature to read date tags from groups EXIF/XMP/QuickTime/Track/Media/Matroska/RIFF only (other groups are not reliable).

      • Improved Auto Rename Albums feature to choose the oldest date with time when two or more candidates has the same date but one has time o other not.

      • Updated GPTH to version 5.0.5 (by @Xentraxx & @jaimetur) which includes new features, performance improvements and bugs fixing extracting metadata info from Google Takeouts.

      • ✨ GPTH New Features

        • Added support for Special Folders management such as Archive, Trash, Locked folder. Now those folders are excluded from all album strategies and are moved directly to the output folder.

      • 🚀 GPTH Improvements

        • Moved logic of each Step to step's service module. Now each step has a service associated to it which includes all the logic and interation with other services used by this step.
        • Added percentages to all progress bars.
        • Added Total time to Telemetry Summary in Step 3.
        • Fixed _extractBadPathsFromExifError method to detect from exiftool output bad files with relative paths.
        • Performance Improvements in step_03_merge_media_entities_service.dart.
          • Now grouping method can be easily changed. Internal _fullHashGroup is now used instead of 'groupIdenticalFast' to avoid calculate buckets again.

      • 🐛 Bug Fixes

        • Fixed duplicated files/symlinks in Albums when a file belong to more than 1 album (affected strategies: shortcut, reverse-shortcut & duplicate-copy).
        • Fixed error decoding Exiftool output with UTF-8/latin chars.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

What's Changed

New Contributors

Full Changelog: v3.6.1...v3.6.2

Contributors

WizardOfYendor1, jaimetur, and Xentraxx
Loading
0 Join discussion

v3.6.1

08 Sep 18:15
@jaimetur jaimetur
v3.6.1
1bafaba

Choose a tag to compare

View all tags
v3.6.1

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases+

Release: v3.6.1

  • Release Date: 2025-09-09

    • Main Changes:

    • 🚀 Enhancements:

      • Now Logs, Extracted_dates_metadata.json and Duplcicates.csv files are saved at Output folder by default when Google Takeout Processing feature is detected.

      • Improved FixSymLinks to be case-insensitive when looking for real files of symlinks.

      • Improvements in Analysis Output Statistics to separate Physical and Symlinks files.

      • Show GPTH Tool and EXIF Tool version in Global Configuration settings at the beginning of the tool.

      • Updated GPTH to version 5.0.4 (by @Xentraxx & @jaimetur) which includes new features, performance improvements and bugs fixing extracting metadata info from Google Takeouts.

        • ✨ GPTH New Features

          • New album moving strategy ignore to completely ignore all Albums content. The difference with nothing strategy is that nothing don't create Albums folders but process and move all Albums content into ALL_PHOTOS folder.

        • 🚀 GPTH Improvements

          • Replace all print() functions by logPrint() method from LoggerMixin class. In this way all messages are registered both on screen and also on the logger (and saved to disk if flag --save-log is enabled).
          • All console messages have now a Step prefix to identify from which step or service they come from.
          • Moving Strategies re-defined
          • Included Timeouts on ExifTool operations.
          • Log saving enabled by default. Use flag --no-save-log to disable it.
          • Changed log name from gpth-{version}_{timestamp}.log to gpth_{version}_{timestamp}.log.
          • Added progress bar to Step 3 (Merge Media Entities).
          • Changed default value for flag `--update-creation-time. Now is enabled by default.
          • Smart split in writeBatchSafe: we parse stderr, separate only the conflicting files, retry the rest in a single batch, and write the conflicting ones per-file (without blocking progress). If paths can’t be extracted, we fall back to your original recursive split.

        • 🐛 Bug Fixes

          • Added reverse-shortcut strategy to interactive mode.
          • Fixed some moving strategies that was missing some files in the input folder
          • Fixed exiftool_service.dart to avoid IFD0 pointer references.
          • Fixed exiftool_service.dart to avoid use of -common_args when -@ ARGFILE is used.
          • Fixed PNG management writting XMP instead of EXIF for those files.
          • (ExifToolService): I added -F to the common arguments (_commonWriteArgs). It’s an immediate patch that often turns “Truncated InteropIFD” into a success.
          • (Step 7): If we detect a “problematic” JPEG, we force XMP (CreateDate/DateTimeOriginal/ModifyDate + signed GPS), both when initially building the tags (via _forceJpegXmp) and again on retry when a batch fails and stderr contains Truncated InteropIFD (in-place conversion of those entries with _retagEntryToXmpIfJpeg).

    • 🐛 Bug fixes:

      • Fixed albums_input_folder in Move Albums step within Google Takeout Processing feature.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

Full Changelog: v3.6.0...v3.6.1

Contributors

jaimetur and Xentraxx
Loading
0 Join discussion

v3.6.0

06 Sep 21:32
@jaimetur jaimetur
v3.6.0
9bf8358

Choose a tag to compare

View all tags
v3.6.0

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases+

Release: v3.6.0

  • Release Date: 2025-08-31

  • Main Changes:

    • 🚨 Breaking Changes:

      • Auto-Rename Albums content based now uses as dates separator - instead of . and as group of dates separator -- instead of -. Those separators can be customized using two new parameters (see below).

    • 🌟 New Features:

      • Added new parameter -dateSep, --date-separator <DATE_SEPARATOR> to specify the Dates Separator for the Feature Auto-Rename Albums Content Based.
      • Added new parameter -rangeSep, --range-separator <RANGE_OF_DATES_SEPARATOR> to specify the Range of Dates Separator for the Feature Auto-Rename Albums Content Based.
      • Added new parameter -gpthLog, --gpth-log to Save GPTH log messages into a log file within Logs folder.

    • 🚀 Enhancements:

      • Improvements in Auto Rename Albums Folders feature to Compute the best 'oldest_date' per file using the following priority:

        1. date_dict
        2. EXIF native (piexif)
        3. EXIF exiftool
        4. Filesystem ctime
        5. Filesystem mtime

      • Updated GPTH to version 5.0.2 (by @Xentraxx & @jaimetur) which includes new features, performance improvements and bugs fixing extracting metadata info from Google Takeouts.

        • ✨ GPTH New Features

          • Support for 7zip and unzip extractors (if found in your system). This is because the native extractor does not extract properly filenames or dirnames with UTF-8/latin1 chars.
          • Support new Extra files from Google Takeout with following suffixes: -motion, -animation, -collage.
          • New flag --keep-input to Work on a temporary sibling copy of --input (suffix _tmp), keeping the original untouched.
          • New flag --keep-duplicates to keep duplicates files in _Duplicates subfolder within output folder.
          • New flag --save-log to save log messages into disk file.
          • Created GitHub Action build-and-create-release.yml to Automatically build all binaries, create new release (stable or pre-release), update it wiht the release-notes and upload the binaries to the new release.
          • Step 8 (Update creation time) is now multi-platform. Also update creation date for physical files and symlinks on linux/macos.

        • 🚀 GPTH Improvements

          • Created a single package gpth-lib with all the exported modules for an easier way to manage imports and refactoring.
          • Added new flag fallbackToExifToolOnNativeMissin GlobalConfigService Class to specify if we want to fallback to ExifTool on Native EXIF reader fail. (Normally if Native fails is because EXIF is corrupt, so fallback to ExifTool does not help).
          • Added new flag enableExifToolBatchin GlobalConfigService Class to specify if we want to enable/disable call ExifTool with batches of files instead of one call per file (this speed-up a lot the EXIF writting time with ExifTool).
          • Added new flag maxExifImageBatchSizein GlobalConfigService Class to specify the maximum number of Images for each batch passed in any call to ExifTool.
          • Added new flag maxExifVideoBatchSizein GlobalConfigService Class to specify the maximum number of Videos for each batch passed in any call to ExifTool.
          • Added new flag forceProcessUnsupportedFormatsin GlobalConfigService Class to specify if we want to forze process unsupported format such as .AVI, .MPGor .BMP files with ExifTool.
          • Added new flag silenceUnsupportedWarningsin GlobalConfigService Class to specify if we want to recive or silence warnings due to unsupported format on ExifTool calls.
          • Added new flag enableTelemetryInMergeMediaEntitiesStepin GlobalConfigService Class to enable/disable Telemetry in Step 3: Merge Media Entities.
          • Code Structure refactored for a better understanding and easier way to find each module.
          • Code Refactored to isolate the execution logic of each step into the .execute() function of the step's class. In this way the media_entity_collection module is much clearer and easy to understand and maintain.
          • Adapted all methods to work with this new structure
          • Homogenized logs for all steps.
          • New code re-design to include a new MediaEntity model with the following attributes:
            • albumsMap: List of AlbumsInfo obects, where each object represent the album where each file of the media entity have been found. This List which can contain many usefull info related to the Album.
            • dateTaken: a single dataTaken for all the files within the entity
            • dateAccuracy: a single dateAccuracy for all the files within the entity (based on which extraction method have been used to extract the date)
            • dateTimeExtractionMethod: a single dateTimeExtractionMethod for all the files within the entity (method used to extract the dataTaken assigned to the entity)
            • partnerShared: true if the entity is partnerShared
            • primaryFile: contains the best ranked file within all the entity files (canonical first, then secondaries ranked by lenght of basename, then lenght of pathname)
            • secondaryFiles: contains all the secondary files in the entity
            • duplicatesFiles: contains files which has at least one more file within the entity in the same folder (duplicates within folder)
          • Created internal/external methods for Class MediaEntity for an easy utilization.
          • All modules have been adapted to the new MediaEntity structure.
          • All Tests have been adapted to the new MediaEntity structure.
          • Removed files attribute from MediaEntity Class.
          • Merged media_entity_moving_strategy.dart module with media_entity_moving_service.dart module and now it is stored under lib/steps/step_06_moving_files/services folder.
          • New behaviour during Find Duplicates step:
            • Now, all identical content files are collected within the same MediaEntity.
              • In a typical Takeout, you might have the same file within Photos from yyyy folder and within one or more Album folder
              • So, both of them are collected within the same entity and will not be considered as duplicated because one of them could have associated json and the others not
              • So, we should extract dates for all the files within the same media entity.
            • If one media entity contains two or more files within the same folder, then this is a duplicated file (based on content), even if they have different names, and the tool will remove the worst ranked duplicated file.
          • Moved Write EXIF step to Step 7 (after Move Files step) in order to write EXIF data only to those physical files in output folder (skipping shortcuts).
            • This changed was needed because until Step 6 (based on the selected album strategy), don't create the output physical files, we don't know which files need EXIF write.
            • With this change we reduce a lot the number of EXIF files to write because we can skip writing EXIF for shortcut files created by shorcut or reverse-shortcut strategy, but also we can skip all secondaryFiles if selected strategy is None or Json.
            • The only strategy that has no benefit from this change is duplicate-copy, because in this strategy all files in output folder are physical files and all of them need to have EXIF written.
          • Renamed Step 3: Remove Duplicates to Step 3: Merge Media Entities because this represents much better the main purpose of this step.
          • Performance Optimization in Step 3: Merge Media Entities.
          • Step 3: Merge Media Entities now only consider within-folder duplicates. And take care of the primaryFile/secondaryFiles based on a ranking for the rest of the pipeline.
          • Step 7: Write EXIF now take into account all the files in the MediaEntity file except duplicatesFiles and files with isShortcut=true attribute.
          • Step 6: Move Files now manage hardlinks/juntions as fallback of native shorcuts using API to WindowsSymlinkService when no admin rights are granted.
          • Step 8: Update Creation Timenow take into account all the files in the MediaEntity file except duplicatesFiles.
          • Step 8: Update Creation Timenow update creation time also for shortcuts.
          • Improvements on Statistics results.
            • Added more statistics to Step 3: Remove Duplicate
            • Added more statistics to Step 6: Move Files
            • Added more statistics to Step 8: Update Creation Time.
            • Total execution time is now shown as hh:mm:ss instead of only minutes.

        • 🐛 GPTH Bug Fixes

          • Fixed #65: Now all supported media files are moved from input folder to output folder. So after running GPTH input folder should only contain .json files and unsupported media types.
          • Fixed #76: Now interactive mode ask for album strategy.
          • Changed zip_extraction_service.dart to support extract UTF-8/latin1 chars on folder/files names.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

  • [Download x64 vers...
Read more

Contributors

jaimetur and Xentraxx
Loading
0 Join discussion

v3.5.2

26 Aug 17:25
@jaimetur jaimetur
v3.5.2
ed132fa
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v3.5.2

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases+

Release: v3.5.2

  • Release Date: 2025-08-24

  • Main Changes:

    • 🚀 Enhancements:

      • Separate Steps for Analyze Input Takeout files and Analyze Output files in feature Google Takeout Fixing. Now the Analyzer for the Input Takeout folder is executed after Pre-Processing steps, in this way the extracted dates JSON dictionary will match for all input files during GPTH processing phase even if any file was renamed during any pre-processing step.
      • Show dictMiss files in GPTH log to see those files that have not been found in dates dictionary when it was passed as argument using --fileDates

    • 🐛 Bug fixes:

      • Fixed a bug in GPTH in JSON Matcher service when the JSON filename lenght is longer than 51 chars. In those cases the tool was trying to find a truncated JSON variant and never try to match with the full filename, but PhotoMigrator fixes Truncations during Pre-process, so it was never found.
      • Fixed a bug in GPTH Step 7 (Move Files) that progress bar was not showing the correct number of operations.

    • 📚 Documentation:

      • Updated documentation with all changes.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

What's Changed

Full Changelog: v3.5.1...v3.5.2

Contributors

jaimetur
Loading
0 Join discussion

v3.5.1

22 Aug 18:41
@jaimetur jaimetur
v3.5.1
a1dfb16
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v3.5.1

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases

Release: v3.5.1

  • Release Date: 2025-08-20

  • Main Changes:

    • 🚀 Enhancements:

      • Extracted Dates JSON now contains all Dates in Local UTC format.
      • Extracted Dates JSON now includes ExecutionTimestamp Tag for reference.
      • Extracted Dates JSON Tags renamed for a better understanding.
      • Enhancements in extract_dates() function to avoid guess date from file path when the file path cotains the execution timestamp.
      • Included support in GPTH to use the Extracted Dates JSON dictionary. This will speed-up GPTH Date extraction a lot.
      • Updated GPTH to version 4.3.0 (by @Xentraxx & @jaimetur) which includes new features, performance improvements and bugs fixing extracting metadata info from Google Takeouts.

    • 🚀 Enhancements in GPTH Tool:

      • Improved non-zero exit code quitting behaviour - Now with nice descriptive error messages because I was tired of looking up what is responsible for a certain exit code.

      • Step 4 (Extract Dates) & 5 (Write EXIF) Optimization

        • ⚡ Performance
          • Step 4 (READ-EXIF) now support --fileDates flag to provide a JSON dictionary with the Extracted dates per file (PhotoMigrator creates this file and can now be used by GPTH Tool).
          • Step 4 (READ-EXIF) now uses batch reads and a fast native mode, with ExifTool only as fallback → about 3x faster metadata extraction.
          • Step 5 (WRITE-EXIF) supports batch writes and argfile mode, plus native JPEG writers → up to 5x faster on large collections.
        • 🔧 API
          • Added batch write methods in ExifToolService.
          • Updated MediaEntityCollection to use new helpers for counting written tags.
        • 📊 Logging
          • Statistics are clearer: calls, hits, misses, fallback attempts, timings.
          • Date, GPS, and combined writes are reported separately.
          • Removed extra blank lines for cleaner output.
        • 🧪 Testing
          • Extended mocks with batch support and error simulation.
          • Added tests for GPS writing, batch operations, and non-image handling.
        • ✅ Benefits
          • Much faster EXIF processing with less ExifTool overhead.
          • More reliable and structured API.
          • Logging is easier to read and interpret.
          • Stronger test coverage across edge cases.

      • Step 6 (Find Albums) Optimization

        • ⚡ Performance
          • Replaced _groupIdenticalMedia with _groupIdenticalMediaOptimized.
            • Two-phase strategy:
              • First group by file size (cheap).
              • Only hash files that share the same size.
            • Switched from readAsBytes() (full memory load) to streaming hashing with md5.bind(file.openRead()).
            • Files are processed in parallel batches instead of sequentially.
            • Concurrency defaults to number of CPU cores, configurable via maxConcurrent.
        • 🔧 Implementation
          • Added an in-memory hash cache keyed by (path|size|mtime) to avoid recalculating.
            • Introduced a custom semaphore to limit concurrent hashing and prevent I/O overload.
            • Errors are handled gracefully: unprocessable files go into dedicated groups without breaking the process.
        • ✅ Benefits
          • Processing time reduced from 1m20s → 4s on large collections.
            • Greatly reduced memory usage.
            • Scales better on multi-core systems.
            • More robust and fault-tolerant album detection.

    • 🐛 Bug Fixes in GPTH Tool:

      • Changed exif tags to be utilized
        • Before we used the following lists of tags in this exact order to find a date to set:
          • Exiftool reading: 'DateTimeOriginal', 'MediaCreateDate', 'CreationDate', 'TrackCreateDate', 'CreateDate', 'DateTimeDigitized', 'GPSDateStamp' and 'DateTime'.
          • Native dart exif reading: 'Image DateTime', 'EXIF DateTimeOriginal', 'EXIF DateTimeDigitized'.
        • Some of those values are prone to deliver wrong dates (e.g. DateTimeDigitized) and the order did not completely make sense. We therefore now read those tags and the oldest DateTime we can find:
          • Exiftool reading: 'DateTimeOriginal','DateTime','CreateDate','DateCreated','CreationDate','MediaCreateDate','TrackCreateDate','EncodedDate','MetadataDate','ModifyDate'.
          • Native dart exif reading: same as above.
      • Fixed typo in partner sharing - Functionality was fundamentally broken due to a typo.
      • Fixed small bug in interactive mode in the options of the limit filezise dialogue
      • Fixed unzipping through command line by automatically detecting if input directory contains zip files

    • 🐛 Bug fixes:

      • Fixed a bug when guessed date from filepath extract the same date than TIMESTAMP (if the path contains the current TIMESTAMP).

    • 📚 Documentation:

      • Updated documentation with all changes.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

What's Changed

Full Changelog: v3.5.0...v3.5.1

Contributors

jaimetur and Xentraxx
Loading
0 Join discussion

v3.5.0

30 Jul 22:48
@jaimetur jaimetur
v3.5.0
42652cb
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v3.5.0

🗓️ CHANGELOG

Planned Roadmap for the following releases
Changelog for the past releases

Release: v3.5.0

  • Release Date: 2025-07-30

  • Main Changes:

    • 🚀 Enhancements:

      • Updated GPTH to version 4.1.0 (by @Xentraxx) which includes several new features, improvements and bugs fixing extracting metadata info from Google Takeouts.

    • ✨ New Features in GPTH Tool:

      • Partner Sharing Support - Added --divide-partner-shared flag to separate partner shared media from personal uploads into dedicated PARTNER_SHARED folder (Issue #56)
        • Automatically detects partner shared photos from JSON metadata (googlePhotoOrigin.fromPartnerSharing)
        • Creates separate folder structure while maintaining date division and album organization
        • Works with all album handling modes (shortcut, duplicate-copy, reverse-shortcut, json, nothing)
        • Preserves album relationships for partner shared media
      • Added folder year date extraction strategy - New fallback date extractor that extracts year from parent folder names like "Photos from 2005" when other extraction methods fail (Issue #28)
      • Centralized concurrency management - Introduced ConcurrencyManager for consistent concurrency calculations across all services, eliminating hardcoded multipliers scattered throughout the codebase
      • Displaying version of Exiftool when found - Instead of just displaying that Exif tool was found, we display the version now as well.

    • 🚀 Performance Improvements in GPTH Tool:

      • EXIF processing optimization - Native exif_reader library integration for 15-40% performance improvement in EXIF data extraction
        • Uses fast native library for supported formats (JPEG, TIFF, HEIC, PNG, WebP, AVIF, JXL, CR3, RAF, ARW, DNG, CRW, NEF, NRW)
        • Automatic fallback to ExifTool for unsupported formats or when native extraction fails
        • Centralized MIME type constants verified against actual library source code
        • Improved error logging with GitHub issue reporting guidance when native extraction fails
      • GPS coordinate extraction optimization - Dedicated coordinate extraction service with native library support
        • 15-40% performance improvement for GPS-heavy photo collections
        • Clean architectural separation between date and coordinate extraction
        • Centralized MIME type support across all EXIF processing operations
      • Significantly increased parallelization - Changed CPU concurrency multiplier from ×2 to ×8 for most operations, dramatically improving performance on multi-core systems
      • Removed concurrency caps - Eliminated .clamp() limits that were artificially restricting parallelization on high-core systems
      • Platform-optimized concurrency:
        • Linux: Improved from CPU cores + 1 to CPU cores × 8 (massive improvement for Linux users)
        • macOS: Improved from CPU cores + 1 to CPU cores × 6
        • Windows: Maintained at CPU cores × 8 (already optimized)
      • Operation-specific concurrency tuning:
        • Hash operations: CPU cores × 4 (balanced for CPU + I/O workload)
        • EXIF/Metadata: CPU cores × 6 (I/O optimized for modern SSDs)
        • Duplicate detection: CPU cores × 6 (memory intensive, conservative)
        • Network operations: CPU cores × 16 (high for I/O waiting)
      • Adaptive concurrency scaling - Dynamic performance-based concurrency adjustment that scales up to ×24 for high-performance scenarios

    • 🐛 Bug Fixes in GPTH Tool:

      • Fixed memory exhaustion during ZIP extraction - Implemented streaming extraction to handle large ZIP files without running out of memory
      • Fixed atomic file operations - Changed to atomic file rename operations to resolve situations where only the json was renamed in file extension correction (Issue #60)
      • Fixed album relationship processing - Improved album relationship service to handle edge cases properly (Issue #61)
      • Fixed interactive presenter display - Corrected display issue in interactive mode (Issue #62)
      • Fixed date division behavior for albums - The --divide-to-dates flag now only applies to ALL_PHOTOS folder, leaving album folders flattened without date subfolders (Issue #55)
      • Reaorganised ReadMe for a more intuitive structure - First Installation, then prerequisites and then the quickstart.
      • Step 8 now also uses a progress bar instead of simple print statements
      • Supressed some unnecessary ouput

    • 📚 Documentation:

      • Updated documentation with all changes.

💾 Download

Download the tool either for Linux, MacOS or Windows (for both x64 and arm64 architectures) or Docker version (platform & architecture independent) as you prefer, directly from following links:

Linux::

Mac OS:

Windows:

Docker Launcher:

What's Changed

Full Changelog: v3.4.4...v3.5.0

Contributors

jaimetur and Xentraxx
Loading
0 Join discussion