Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ffmpeg] Update to 7.0. #38011

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open

[ffmpeg] Update to 7.0. #38011

wants to merge 25 commits into from

Conversation

Sibras
Copy link
Contributor

@Sibras Sibras commented Apr 6, 2024

Update ffmpeg to 7.0
Fixes #37888

  • [*] Changes comply with the maintainer guide.
  • [*] SHA512s are updated for each updated download.
  • [*] The "supports" clause reflects platforms that may be fixed by this new version.
  • [*] Any fixed CI baseline entries are removed from that file.
  • [*] Any patches that are no longer applied are deleted from the port's directory.
  • [*] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
  • [*] Only one version is added to each modified port's versions file.
@MonicaLiu0311 MonicaLiu0311 added the category:port-update The issue is with a library, which is requesting update new revision label Apr 7, 2024
@Sibras
Copy link
Contributor Author

Sibras commented Apr 7, 2024

Most of the CI failures are due to avcpp or aubio failing with the latest ffmpeg. Unfortunately these projects currently dont support ffmpeg 7.0 and thus fail (which is a common theme with ffmpeg updates within vcpkg).

@MonicaLiu0311
Copy link
Contributor

Is there any new progress?

@MonicaLiu0311 MonicaLiu0311 marked this pull request as draft May 28, 2024 07:42
@@ -94,7 +95,7 @@ if(VCPKG_DETECTED_CMAKE_C_COMPILER)
get_filename_component(CC_filename "${VCPKG_DETECTED_CMAKE_C_COMPILER}" NAME)
set(ENV{CC} "${CC_filename}")
string(APPEND OPTIONS " --cc=${CC_filename}")
#string(APPEND OPTIONS " --host_cc=${CC_filename}") ffmpeg not yet setup for cross builds?
string(APPEND OPTIONS " --host_cc=${CC_filename}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no host_cc in vcpkg.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

host_cc is a command line arg passed to ffmpeg configure script. Due to the way the portfile is calling ffmpegs configure (calld it as cross compile) it fails without this command option

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will also fail with this option except for "almost-native" cross builds.
vcpkg allows you to tell the compiler and flags for the target triplet (e.g. android, ios).
vcpkg allows you to know the host triplet and its install prefix.
vcpkg doesn't allow you to tell the compiler and flags for the host triplet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR vcpkg_configure_make turns the host cc (CC_FOR_BUILD) into a no-op for VCPKG_CROSSCOMPILING.

Copy link
Contributor Author

@Sibras Sibras Jun 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately someone set the cross compile flag for ffmpeg as always on in PR #23001 (see portfile line 459). I have no idea why or what they were trying to fix but I have no way of testing all the various triplets that this effects in order to work out how to disable it again.
Due to changes in ffmpegs internal configure script in 7.0 it now does additional checks on the host compiler when the cross compile flag is passed to it. This is why the line had to be uncommented as its required to pass those checks. The actual compilation though functions the same as previously.

Copy link
Contributor Author

@Sibras Sibras Jun 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR vcpkg_configure_make turns the host cc (CC_FOR_BUILD) into a no-op for VCPKG_CROSSCOMPILING.

Unfortunately the ffmpeg port doesnt use vcpkg_configure_make

It uses vcpkg_execute_required_process to run a custom bash script. The --host_cc command line option is for this bash script to pass through to ffmpegs internal configure script

Comment on lines 8 to 10
enabled libfdk_aac && { check_pkg_config libfdk_aac fdk-aac "fdk-aac/aacenc_lib.h" aacEncOpen ||
- { require libfdk_aac fdk-aac/aacenc_lib.h aacEncOpen -lfdk-aac &&
+ { require libfdk_aac fdk-aac/aacenc_lib.h aacEncOpen -lfdk-aac -lm -lstdc++ &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I know this is the existing patch, but...)
Can't we make check_pkg_config libfdk_aac ... do the right thing?
Encoding -lstdc++ is known to be wrong for a number of systems.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is only called when check_pkg_config fails (cant find a pkgconfig for instance). FFmpegs configure script will later strip -lstdc++ (and -lm) on systems that dont support it which is why it doesnt cause errors.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#39077:

  • Export actual c++ link libraries for fdk-aac via pkg-config. (Same pattern as lerc, geos.)
  • Setup pkg-config for ffmpeg.
Comment on lines +9 to +12
-check_host_cflags_cc -std=$stdc ctype.h "__STDC_VERSION__ >= 201112L" ||
- check_host_cflags_cc -std=c11 ctype.h "__STDC_VERSION__ >= 201112L" || die "Host compiler lacks C11 support"
+check_host_cflags_cc -std=$stdc ctype.h "__STDC_VERSION__ >= 201112L" ||
+ check_host_cflags_cc -std=c11 ctype.h "__STDC_VERSION__ >= 201112L"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a change in the first line.
I wonder about the second line: Is the check wrong for osx, or doesn't it get the required flags?
But we dont have a host_cc anyways, so probably the latter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned previously the hostcc is passed via command line option (and is just set to regular cc).
Im not sure why git diff created the patch the way it did but the only change is the removal of the "die" line as it fails in the osx CI otherwise (I dont have osx to local debug why)

@Sibras
Copy link
Contributor Author

Sibras commented Jun 1, 2024

Is there any new progress?

avcpp has been updated and now works with ffmepg 7 but aubio still fails as it hasnt been updated. This was also a blocker for the 6 update as it doesnt get updated (the last aubio release was over 5 years ago) so without a patch for aubio its still failing in the CI

@Neumann-A
Copy link
Contributor

does ffmpeg have a migration guide somewhere? Maybe its easy to patch.

@Sibras
Copy link
Contributor Author

Sibras commented Jun 1, 2024

Does this help: https://gitlab.archlinux.org/archlinux/packaging/packages/aubio/-/blob/main/ffmpeg7.patch?ref_type=heads

So it turns out aubio port wasnt using release versions anymore anyway so i bumbped to latest commit on master and then added that patch and it seems now to be good

vicroms pushed a commit that referenced this pull request Jun 6, 2024
…g-config (#39077)

- Setup and use pkg-config for ffmpeg dependencies.
#38011 (comment).
- Export actual c++ link libraries for fdk-aac via pkg-config. (Same
pattern as lerc, geos.)
- Rectify link libraries in pkg-config  for alsa, libsrt, snappy, x265.
- Burn-in dllimport for libsrt and x265.
- Pass detected STRIP to ffmpeg. Fixes
#36852.
@julianxhokaxhiu
Copy link
Contributor

Is it possible to add support for D3D12VA as well as you're into this PR? I was trying to approach it on a different PR ( see #39322 ) but we figured out it's a 7.0+ feature. The diff looks like this basically:

diff --git a/ports/ffmpeg/portfile.cmake b/ports/ffmpeg/portfile.cmake
index 72055b063..484ca01a5 100644
--- a/ports/ffmpeg/portfile.cmake
+++ b/ports/ffmpeg/portfile.cmake
@@ -48,9 +48,9 @@ if(VCPKG_TARGET_IS_MINGW)
 elseif(VCPKG_TARGET_IS_LINUX)
     string(APPEND OPTIONS " --target-os=linux --enable-pthreads")
 elseif(VCPKG_TARGET_IS_UWP)
-    string(APPEND OPTIONS " --target-os=win32 --enable-w32threads --enable-d3d11va --enable-mediafoundation")
+    string(APPEND OPTIONS " --target-os=win32 --enable-w32threads --enable-d3d11va --enable-d3d12va --enable-mediafoundation")
 elseif(VCPKG_TARGET_IS_WINDOWS)
-    string(APPEND OPTIONS " --target-os=win32 --enable-w32threads --enable-d3d11va --enable-dxva2 --enable-mediafoundation")
+    string(APPEND OPTIONS " --target-os=win32 --enable-w32threads --enable-d3d11va --enable-d3d12va --enable-dxva2 --enable-mediafoundation")
 elseif(VCPKG_TARGET_IS_OSX)
     string(APPEND OPTIONS " --target-os=darwin --enable-appkit --enable-avfoundation --enable-coreimage --enable-audiotoolbox --enable-videotoolbox")
 elseif(VCPKG_TARGET_IS_IOS)

Thank you in advance 🙏

@kadirlua
Copy link
Contributor

ffmpeg 7.0.1 has been released. It would be nice adding support for 7.0.1 while you already work on this. @Sibras

@Sibras
Copy link
Contributor Author

Sibras commented Jun 30, 2024

Is it possible to add support for D3D12VA as well as you're into this PR? I was trying to approach it on a different PR ( see #39322 ) but we figured out it's a 7.0+ feature. The diff looks like this basically:

Thank you in advance 🙏

Done

@Sibras
Copy link
Contributor Author

Sibras commented Jun 30, 2024

ffmpeg 7.0.1 has been released. It would be nice adding support for 7.0.1 while you already work on this. @Sibras

7.0.1 should be an easy update. Just need to get ther CI clear with 7.0 first

@Sibras Sibras marked this pull request as ready for review July 7, 2024 06:00
"port-version": 1,
"description": "Aubio is a tool designed for the extraction of annotations from audio signals. Its features include segmenting a sound file before each of its attacks, performing pitch detection, tapping the beat and producing midi streams from live audio.",
"homepage": "https://github.com/aubio/aubio",
"license": "GPL-3.0-or-later",
"supports": "!xbox",
"supports": "!xbox & !osx",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aubio is healthy on osx. ffmpeg and its dependencies are not. Until #39703.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aubio was failing to link on osx (and I have no way to debug osx). It was the only one failing to link so I disabled it. If #39703 fixes it then ill revert the change

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#39703 fixes it. ffmpeg and its vcpkg port hardcode assumptions about libstdc++ which break libc++ systems such as osx and android.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The actuall error for aubio with ffmepg 7 was missing a framework link option causing a link error (was missing opengl and one other that i cant remember) which wasnt related to libstc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this was an issue, too. libstdc++ wasn't the only problem with the current ffmpeg, and I wouldn't be surprised if 7.0 adds other challenges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:port-update The issue is with a library, which is requesting update new revision
7 participants