HEIC Files Missing in Photoprism #4034

Jhabb7 · 2024-01-25T22:51:58Z

Jhabb7
Jan 25, 2024

Hi all, I have been reading posts for the better part of the last 2 days to try and understand how Photoprism handles .heic files. Essentially, I have found that if I convert a file to .jpeg then import, this solves the problem. However, I am seeing in the logs that photoprism is converting files, so I'm not sure what I'm missing.

Summary of problem:
I am migrating from Google Photos via Takeout extract, and webdav file save to photoprism's originals folder. Upon doing so (35k files and 15k pictures in photoprism with addition 5k heic photos) I do not see any .heic photos in the library. This is about 1/4 of the library missing. They are present in the originals folder and I can view them remotely via windows explorer, but photo prism UI does not show them in search, or in the UI's originals folder.

Log for conversion:

2024-01-25 16:33:14.050433-05:00time="2024-01-25T16:33:14-05:00" level=info msg="import: moving main heic file IMG_5574.HEIC to 2020/03/20200321_003854_3D4F2C7A.heic"

2024-01-25 16:33:14.062040-05:00time="2024-01-25T16:33:14-05:00" level=info msg="convert: converting 20200321_003854_3D4F2C7A.heic to 20200321_003854_3D4F2C7A.heic.jpg (heif-convert)"

2024-01-25 16:33:14.084725-05:00time="2024-01-25T16:33:14-05:00" level=error msg="import: Input file '/photoprism/originals/2020/03/20200321_003854_3D4F2C7A.heic' is a JPEG image in 2020/03/20200321_003854_3D4F2C7A.heic (convert to jpeg)"

1. What is not working as documented?
HEIC Files are displayed in photoprism

2. How can we reproduce it?
Add HEIC file to original folder -> run complete scan
OR
Add HEIC file to import foler -> import

3. What behavior do you expect?
I expect the HEIC image that is visible to webdav in the original directory (whether uploaded there or imported to there) to display alongside the other images

4. What could be the cause of your problem?
conversion is not occurring, these files are somehow ignored, etc.

6. Can you provide us with example files for testing, error logs, or screenshots?

2024-01-25 16:33:13.407161-05:00time="2024-01-25T16:33:13-05:00" level=info msg="processing upload..."
2024-01-25 16:33:13.456625-05:00time="2024-01-25T16:33:13-05:00" level=info msg="import: found no .ppignore file"
2024-01-25 16:33:13.675800-05:00time="2024-01-25T16:33:13-05:00" level=info msg="import: added folder /assets/storage/users/us7q86b2a72hiz07/upload/sessem6mhyfac5q038"
2024-01-25 16:33:13.835391-05:00time="2024-01-25T16:33:13-05:00" level=info msg="media: IMG_5574.HEIC was taken at 2020-03-21 00:38:54 +0000 UTC (meta)"
2024-01-25 16:33:14.050306-05:00time="2024-01-25T16:33:14-05:00" level=info msg="folder: added album 'March 2020' (path:2020/03 public:true)"
2024-01-25 16:33:14.050375-05:00time="2024-01-25T16:33:14-05:00" level=info msg="import: created folder /2020/03"
2024-01-25 16:33:14.050433-05:00time="2024-01-25T16:33:14-05:00" level=info msg="import: moving main heic file IMG_5574.HEIC to 2020/03/20200321_003854_3D4F2C7A.heic"
2024-01-25 16:33:14.062040-05:00time="2024-01-25T16:33:14-05:00" level=info msg="convert: converting 20200321_003854_3D4F2C7A.heic to 20200321_003854_3D4F2C7A.heic.jpg (heif-convert)"
2024-01-25 16:33:14.084725-05:00time="2024-01-25T16:33:14-05:00" level=error msg="import: Input file '/photoprism/originals/2020/03/20200321_003854_3D4F2C7A.heic' is a JPEG image in 2020/03/20200321_003854_3D4F2C7A.heic (convert to jpeg)"
2024-01-25 16:33:14.576219-05:00time="2024-01-25T16:33:14-05:00" level=info msg="upload: deleted empty folder /assets/storage/users/us7q86b2a72hiz07/upload/sessem6mhyfac5q038"
2024-01-25 16:33:14.576255-05:00time="2024-01-25T16:33:14-05:00" level=info msg="upload: imported 2 files"
2024-01-25 16:33:14.576269-05:00time="2024-01-25T16:33:14-05:00" level=warning msg="moments: already running"
2024-01-25 16:33:14.576279-05:00time="2024-01-25T16:33:14-05:00" level=info msg="upload has been processed"
2024-01-25 16:33:27.339551-05:00time="2024-01-25T16:33:27-05:00" level=info msg="folder: added album 2020 (path:2020 public:true)"
2024-01-25 16:53:18.973192-05:00time="2024-01-25T16:53:18-05:00" level=info msg="1 files uploaded in 0 s"

7. Which software versions do you use?

(a) PhotoPrism Architecture & Build Number: Build [231128-f48ff16ef] This was installed via truecharts.

(b) Operating System Types & Versions: TrueNAS-SCALE-23.10.1

(c) Browser Types & Versions: Safari, Firefox on MacOS

(d) Ad Blockers, Browser Plugins, and/or Firewall Software: None

8. On what kind of device is PhotoPrism installed?
(a) Device / Processor Type: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz

(b) Physical Memory & Swap Space in GB : 16GB RAM + 16GB swap

(c) Storage Type: 1TB HDD

(d) Anything else that might be helpful to know?

N/A

Do you use a Reverse Proxy, Firewall, VPN, or CDN?
No

Answered by lastzero

Jan 26, 2024

see https://github.com/photoprism/photoprism-contrib/tree/main/scripts/Batch%20Rename

View full answer

lastzero · 2024-01-26T01:46:14Z

lastzero
Jan 26, 2024
Maintainer

What does the error message say?

8 replies

lastzero Jan 26, 2024
Maintainer

Check (again) the logs you posted 🧐

Jhabb7 Jan 26, 2024
Author

Thank you! I am blind haha

Okay, what does that "moving IMG_5574" mean? Is that a simple rename?

The image that is not displaying is 5574, I assumed the longer named one was its conversion, followed by a failure.

By all accounts the file type is HEIC in windows explorer > properties, so it is very shocking that it says it is jpg.

Thanks again!

Jhabb7 Jan 26, 2024
Author

@lastzero I used imageMagick to confirm what you were saying. Do you have any tips for mass searching the directory and converting photos to their appropriate file types?

Or is there a way to get photoprism to handle this itself?

Thanks again!

lastzero Jan 26, 2024
Maintainer

see https://github.com/photoprism/photoprism-contrib/tree/main/scripts/Batch%20Rename

Answer selected by Jhabb7

Jhabb7 Jan 26, 2024
Author

@lastzero this is quite novice, but how do I run that script?

Thank you for your patience lol

lastzero Jan 26, 2024
Maintainer

wget https://raw.githubusercontent.com/photoprism/photoprism-contrib/main/scripts/Batch%20Rename/heic2jpg.sh
chmod a+rx heic2jpg.sh
./heic2jpg.sh

Jhabb7 Jan 26, 2024
Author

Thank you, you're amazing. Wondering if this could me a feature request in the future.

For anyone following this thread in the future, please use ImageMagick identify to confirm your image type

alve89 · 2024-05-18T09:20:41Z

alve89
May 18, 2024

I'd like to step in as well because I don't understand it.

What I did:

Pushing two photo files from my iPhone (iOS 17.4.1) via PhotoSync to PhotoPrism (Build 240420-ef5f14bc4, installed from tteck on Proxmox VE 8.2.2):

root@photoprism:/opt/photoprism# ls photos/import
IMG_4161.HEIC  IMG_4161.MOV  IMG_4162.JPG

Importing the photos from within PhotoPrism > Library > Import:

root@photoprism:/opt/photoprism# ls photos/originals/2024/05
20240517_193335_E2E87CA7.heic  20240517_193335_E2E87CA7.mov  20240518_071626_AC198761.jpg

root@photoprism:/opt/photoprism# ls storage/sidecar/2024/05
20240518_071626_AC198761.yml

Checking PhotoPrism > Library > Originals:
There's one photo listed: 2024/05/20240518_071626_AC198761.jpg

Since this script converts IMG_*.HEIC to IMG_*.JPG, but there are no photos matching this pattern in originals, nothing is renamed.
If I run the script in import, neither of the photos is renamed (I added three echo's to the script - before, within and after the for-loop):

root@photoprism:/opt/photoprism/photos/import# ls
IMG_4161.HEIC  IMG_4161.MOV  IMG_4162.JPG  heic2jpg.sh

root@photoprism:/opt/photoprism/photos/import# sh heic2jpg.sh
Start renaming
Finished renaming

This is my config in /opt/photoprism/config/.env:

PHOTOPRISM_AUTH_MODE='password'
PHOTOPRISM_ADMIN_PASSWORD='changeme'
PHOTOPRISM_HTTP_HOST='0.0.0.0'
PHOTOPRISM_HTTP_PORT='2342'
PHOTOPRISM_SITE_CAPTION='PhotoPrism'
PHOTOPRISM_STORAGE_PATH='/opt/photoprism/storage'
PHOTOPRISM_ORIGINALS_PATH='/opt/photoprism/photos/originals'
PHOTOPRISM_IMPORT_PATH='/opt/photoprism/photos/import'

What am I doing wrong? My "simple" target is to import .HEIC files from my iPhone to PhotoPrism, keep the .HEIC files (like the RAW files of my camera) and making a .JPG conversion out of it.

1 reply

lastzero May 18, 2024
Maintainer

see #4073 (reply in thread)

alve89 · 2024-05-18T13:51:33Z

alve89
May 18, 2024

Thank you for your reply!

I don't really understand what the current state and the problem is... 🙈 Is there a general problem with the libheif library and therefore with the support of .HEIC photos within PhotoPrism?

4 replies

lastzero May 18, 2024
Maintainer

I'd love to help, but it's Saturday and I'm invited to a birthday party right now. Please read the discussion I linked to in full. To summarise, you are using a third party integration (so we don't know anything about it). However, what I do know is that someone using a similar environment has an old script installed (instead of using heif-convert directly) that does not support the quality parameter and therefore HEIC images cannot be indexed. From what you wrote, your problem doesn't seem to be that your HEIC files are JPEGs.

lastzero May 18, 2024
Maintainer

I also recommend taking a look at our Getting Started FAQ to see if you can use our Docker image with LXC/Proxmox instead of a custom version that is not tested/maintained by us in any way:

https://docs.photoprism.app/getting-started/faq/#lxc-images

alve89 May 18, 2024

Of course I don't want to disturb you and you weekend plans! 😊

I read the discussion but it was quite too technical for me...

I also recommend taking a look at our Getting Started FAQ to see if you can use our Docker image with LXC/Proxmox instead of a custom version that is not tested/maintained by us in any way:

https://docs.photoprism.app/getting-started/faq/#lxc-images

I try this one or rather the recommended Docker way.

But I don't understand your last sentence:

From what you wrote, your problem doesn't seem to be that your HEIC files are JPEGs.

That's correct, my problem is not, that my HEIC files are JPEGs. My problem is, that a HEIC file is imported to originals but not converted to JPEG and not converted.

But anyways - enjoy your weekend now! Maybe you can look into this next week. It's not the biggest matter, I could also just import JPEGs.

lastzero May 18, 2024
Maintainer

If there's an existing JPEG Sidecar file with the same name, the HEIC will not be converted as it is assumed that this is the JPEG version of the file. See our User Guide for details.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HEIC Files Missing in Photoprism #4034

{{title}}

Replies: 3 comments 13 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

HEIC Files Missing in Photoprism #4034

Jhabb7 Jan 25, 2024

Replies: 3 comments · 13 replies

lastzero Jan 26, 2024 Maintainer

lastzero Jan 26, 2024 Maintainer

Jhabb7 Jan 26, 2024 Author

Jhabb7 Jan 26, 2024 Author

lastzero Jan 26, 2024 Maintainer

Jhabb7 Jan 26, 2024 Author

lastzero Jan 26, 2024 Maintainer

Jhabb7 Jan 26, 2024 Author

alve89 May 18, 2024

lastzero May 18, 2024 Maintainer

alve89 May 18, 2024

lastzero May 18, 2024 Maintainer

lastzero May 18, 2024 Maintainer

alve89 May 18, 2024

lastzero May 18, 2024 Maintainer

Jhabb7
Jan 25, 2024

Replies: 3 comments 13 replies

lastzero
Jan 26, 2024
Maintainer

lastzero Jan 26, 2024
Maintainer

Jhabb7 Jan 26, 2024
Author

Jhabb7 Jan 26, 2024
Author

lastzero Jan 26, 2024
Maintainer

Jhabb7 Jan 26, 2024
Author

lastzero Jan 26, 2024
Maintainer

Jhabb7 Jan 26, 2024
Author

alve89
May 18, 2024

lastzero May 18, 2024
Maintainer

alve89
May 18, 2024

lastzero May 18, 2024
Maintainer

lastzero May 18, 2024
Maintainer

lastzero May 18, 2024
Maintainer