Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

h264_rkmpp trancoding fails #15095

Open
1 of 3 tasks
ilkondr opened this issue Jan 5, 2025 · 2 comments
Open
1 of 3 tasks

h264_rkmpp trancoding fails #15095

ilkondr opened this issue Jan 5, 2025 · 2 comments

Comments

@ilkondr
Copy link

ilkondr commented Jan 5, 2025

The bug

Hardware transcoding fails on Orangepi 5 board. After quick hacking around it looks like minimal bitrate (of 312) is too low for encoder. Trying manually with higher minimal bitrate finishes successfully. See logs attached.

The OS that Immich Server is running on

Ubuntu 22.04.3 LTS (orangepi5)

Version of Immich Server

v1.123.0

Version of Immich Mobile App

v0

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: rkmpp # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - '2283:2283'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-armnn
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: armnn # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: >-
        pg_isready --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" || exit 1;
        Chksum="$$(psql --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" --tuples-only --no-align
        --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')";
        echo "checksum failure count is $$Chksum";
        [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: >-
      postgres
      -c shared_preload_libraries=vectors.so
      -c 'search_path="$$user", public, vectors'
      -c logging_collector=on
      -c max_wal_size=2GB
      -c shared_buffers=512MB
      -c wal_compression=on
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/media/share/immich/media
# The location where your database files are stored
DB_DATA_LOCATION=/opt/immich/postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

Reproduction steps

  1. Upload video that needs transcoding
  2. see logs
  3. video transcoded by cpu

Relevant log output

[Nest] 7  - 01/05/2025, 12:15:17 PM   ERROR [Microservices:MediaRepository] ffmpeg version 7.0.2-Jellyfin Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu- --toolchain=hardened --enable-cross-compile --enable-rkmpp --enable-rkrga
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fb4180180] All samples in data stream index:id [3:4] have zero duration, stream set to be discarded by default. Override using AVStream->discard or -discard for ffmpeg command.
[h264 @ 0x7fb42c0880] Reinit context to 3840x2160, pix_fmt: yuvj420p
Selecting decoder 'h264_rkmpp' because of requested hwaccel method rkmpp
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'upload/upload/e9b56f91-8bd5-4f04-88e9-e3ebc17d1002/80/2e/802e67e3-fed8-47f2-ba9e-3f784f757ef2.MP4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2022-07-20T10:44:53.000000Z
    firmware        : HD8.01.02.51.00
  Duration: 00:03:40.04, start: 0.000000, bitrate: 100372 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive, left), 3840x2160 [SAR 1:1 DAR 16:9], 100033 kb/s, 25 fps, 25 tbr, 90k tbn (default)
      Metadata:
        creation_time   : 2022-07-20T10:44:53.000000Z
        handler_name    : GoPro AVC
        vendor_id       : [0][0][0][0]
        encoder         : GoPro AVC encoder
        timecode        : 10:44:53:09
  Stream #0:1[0x2](eng): Data: none (tmcd / 0x64636D74) (default)
      Metadata:
        creation_time   : 2022-07-20T10:44:53.000000Z
        handler_name    : GoPro TCD
        timecode        : 10:44:53:09
  Stream #0:2[0x3](eng): Data: bin_data (gpmd / 0x646D7067), 343 kb/s (default)
      Metadata:
        creation_time   : 2022-07-20T10:44:53.000000Z
        handler_name    : GoPro MET
  Stream #0:3[0x4](eng): Data: none (fdsc / 0x63736466), 6 kb/s (default)
      Metadata:
        creation_time   : 2022-07-20T10:44:53.000000Z
        handler_name    : GoPro SOS
[out#0/mp4 @ 0x7fb41b0d80] Adding streams from explicit maps...
[vost#0:0/h264_rkmpp @ 0x7fb42d0180] Created video stream from input stream 0:0
[h264_rkmpp @ 0x7fb42c0880] Found SoC name from device-tree: 'rockchip,rk3588s-orangepi-5 rockchip,rk3588'
[h264_rkmpp @ 0x7fb42c0880] Picked up an existing RKMPP hardware device
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_rkmpp) -> h264 (h264_rkmpp))
[vost#0:0/h264_rkmpp @ 0x7fb42d0180] Starting thread...
[vf#0:0 @ 0x7fb41281a0] Starting thread...
[vist#0:0/h264 @ 0x7fb4050900] [dec:h264_rkmpp @ 0x7fb4190a40] Starting thread...
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fb4080380] Starting thread...
Press [q] to stop, [?] for help
[h264_rkmpp @ 0x7fb42c0880] Noticed an info change
[h264_rkmpp @ 0x7fb42c0880] Configured with size: 3840x2160 | pix_fmt: drm_prime | sw_pix_fmt: nv12
[graph 0 input from stream 0:0 @ 0x7fac090300] w:3840 h:2160 pixfmt:drm_prime tb:1/90000 fr:25/1 sar:1/1 csp:bt709 range:pc
[Parsed_scale_rkrga_0 @ 0x7fac090240] w:3840 h:2160 fmt:nv12 -> w:1280 h:720 fmt:nv12
[graph 0 input from stream 0:0 @ 0x7fac090300] video frame properties congruent with link at pts_time: 0
[h264_rkmpp @ 0x7fb42c1680] Using input frames context (format drm_prime) with h264_rkmpp encoder.
[h264_rkmpp @ 0x7fb42c1680] Rate Control mode is set to AVBR
[h264_rkmpp @ 0x7fb42c1680] Bitrate Target/Min/Max is set to 5000/312/5312
[h264_rkmpp @ 0x7fb42c1680] QP Init/Max/Min/Max_I/Min_I is set to 26/48/0/48/0
[h264_rkmpp @ 0x7fb42c1680] Profile is set to HIGH
[h264_rkmpp @ 0x7fb42c1680] 8x8 Transform is enabled
[h264_rkmpp @ 0x7fb42c1680] Level is set to 51
[h264_rkmpp @ 0x7fb42c1680] Coder is set to CABAC
[h264_rkmpp @ 0x7fb42c1680] Failed to set config: -6
[vost#0:0/h264_rkmpp @ 0x7fb42d0180] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0x7fb41281a0] Error sending frames to consumers: Generic error in an external library
[vf#0:0 @ 0x7fb41281a0] Task finished with error code: -542398533 (Generic error in an external library)
[vf#0:0 @ 0x7fb41281a0] Terminating thread with return code -542398533 (Generic error in an external library)
[vist#0:0/h264 @ 0x7fb4050900] [dec:h264_rkmpp @ 0x7fb4190a40] Decoder returned EOF, finishing
[vost#0:0/h264_rkmpp @ 0x7fb42d0180] Encoder thread received EOF
[vost#0:0/h264_rkmpp @ 0x7fb42d0180] Could not open encoder before EOF
[vost#0:0/h264_rkmpp @ 0x7fb42d0180] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_rkmpp @ 0x7fb42d0180] Terminating thread with return code -22 (Invalid argument)
[vist#0:0/h264 @ 0x7fb4050900] All consumers of this stream are done
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fb4080380] All consumers are done
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fb4080380] Terminating thread with return code 0 (success)
[vist#0:0/h264 @ 0x7fb4050900] [dec:h264_rkmpp @ 0x7fb4190a40] Terminating thread with return code 0 (success)
[out#0/mp4 @ 0x7fb41b0d80] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A
[AVIOContext @ 0x7fb4191800] Statistics: 0 bytes written, 0 seeks, 0 writeouts
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fb4080380] Input file #0 (upload/upload/e9b56f91-8bd5-4f04-88e9-e3ebc17d1002/80/2e/802e67e3-fed8-47f2-ba9e-3f784f757ef2.MP4):
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fb4080380]   Input stream #0:0 (video): 20 packets read (10125734 bytes); 7 frames decoded; 0 decode errors;
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fb4080380]   Total: 20 packets (10125734 bytes) demuxed
[AVIOContext @ 0x7fb4190180] Statistics: 10430277 bytes read, 4 seeks
Conversion failed!

[Nest] 7  - 01/05/2025, 12:15:17 PM   ERROR [Microservices:MediaService] Error occurred during transcoding: ffmpeg exited with code 187: Conversion failed!

[Nest] 7  - 01/05/2025, 12:15:18 PM   ERROR [Microservices:MediaService] Retrying with RKMPP acceleration disabled
[Nest] 7  - 01/05/2025, 12:17:42 PM     LOG [Microservices:MediaService] Successfully encoded 309bee4f-7d29-4c39-b66a-b6f704436eaf



Trying with the same bitrate config manually:

root@orangepi5:/home/orangepi# docker exec -it immich_server /usr/lib/jellyfin-ffmpeg/ffmpeg -f lavfi -i testsrc2=s=1920x1080,format=nv12 -c:v h264_rkmpp -b:v 4M -minrate 312 -maxrate 5M -profile:v high -level 5.1 -g:v 100 -vframes 5000 -y /tmp/foo.mp4
ffmpeg version 7.0.2-Jellyfin Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu- --toolchain=hardened --enable-cross-compile --enable-rkmpp --enable-rkrga
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, lavfi, from 'testsrc2=s=1920x1080,format=nv12':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, nv12, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> h264 (h264_rkmpp))
Press [q] to stop, [?] for help
[h264_rkmpp @ 0x55b1c5f550] Failed to set config: -6
[vost#0:0/h264_rkmpp @ 0x55b1c61cd0] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0x55b1c62380] Error sending frames to consumers: Generic error in an external library
[vf#0:0 @ 0x55b1c62380] Task finished with error code: -542398533 (Generic error in an external library)
[vf#0:0 @ 0x55b1c62380] Terminating thread with return code -542398533 (Generic error in an external library)
[vost#0:0/h264_rkmpp @ 0x55b1c61cd0] Could not open encoder before EOF
[vost#0:0/h264_rkmpp @ 0x55b1c61cd0] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_rkmpp @ 0x55b1c61cd0] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 0x55b1c61450] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A
Conversion failed!



Same result. Now raise minimal bitrate:

root@orangepi5:/home/orangepi# docker exec -it immich_server /usr/lib/jellyfin-ffmpeg/ffmpeg -f lavfi -i testsrc2=s=1920x1080,format=nv12 -c:v h264_rkmpp -b:v 4M -minrate 1312 -maxrate 5M -profile:v high -level 5.1 -g:v 100 -vframes 5000 -y /tmp/foo.mp4
ffmpeg version 7.0.2-Jellyfin Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu- --toolchain=hardened --enable-cross-compile --enable-rkmpp --enable-rkrga
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, lavfi, from 'testsrc2=s=1920x1080,format=nv12':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, nv12, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> h264 (h264_rkmpp))
Press [q] to stop, [?] for help
Output #0, mp4, to '/tmp/foo.mp4':
  Metadata:
    encoder         : Lavf61.1.100
  Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), nv12(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 4000 kb/s, 25 fps, 12800 tbn
      Metadata:
        encoder         : Lavc61.3.100 h264_rkmpp
[out#0/mp4 @ 0x55c58aa450] video:13047KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.026131%
frame=  612 fps=287 q=-0.0 Lsize=   13050KiB time=00:00:24.44 bitrate=4374.2kbits/s speed=11.4x
Exiting normally, received signal 2.


Test successful with higher minimal bitrate

Additional information

No response
@mertalev
Copy link
Contributor

mertalev commented Jan 5, 2025

The bitrate is in bytes if no unit is specified. I think it'd be better if it defaulted to kbps to avoid issues like this.

@ilkondr
Copy link
Author

ilkondr commented Jan 5, 2025

The bitrate is in bytes if no unit is specified. I think it'd be better if it defaulted to kbps to avoid issues like this.

yes it was 5000 in config. setting 5000k solved the problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ilkondr @mertalev