Conda build rpath warnings and out of memory failure

Package Builders conda-build
1

Hello,
I’m trying to build a conda package but it keeps ending with rpath warnings and then exits with a out of memory failure.
I’ve confirmed around 5 GiB of 16 GiB total remains available during the build process. Thus the memory failure doesn’t really apply, and my guess is that the rpath is the problem.
I would be very thankful for any pointers!

Please find below my meta.yml

package:
  name: cryptsetup
  version: "2.7.2"

source:
  url: https://www.kernel.org/pub/linux/utils/cryptsetup/v2.7/cryptsetup-2.7.2.tar.xz
  md5sum: d13fa531c37f05824d67ae92f3c6f73b


requirements:
  host:
    - json-c
    - libdevmapper
    - libgcrypt
    - libssh
    - libtool
    - libxcrypt1
    - lvm2
    - openssl
    - pkgconfig
    - popt
    - util-linux
  build:
    - autoconf
    - automake
    - {{ compiler('c') }}
    - kernel-headers_linux-64

test:
  commands:
    - cryptsetup --help

about:
  home: https://github.com/mbroz/cryptsetup
  license: LGPLv2.1
  doc_url: https://gitlab.com/cryptsetup/cryptsetup#documentation

And my build.sh

#!/bin/bash
./configure --prefix=${PREFIX} --disable-rpath --disable-ssh-token --disable-asciidoc
make
make install
# Copy the [de]activate scripts to $PREFIX/etc/conda/[de]activate.d.
# This will allow them to be run on environment activation.
for CHANGE in "activate" "deactivate"
do
    mkdir -p "${PREFIX}/etc/conda/${CHANGE}.d"
    cp "${RECIPE_DIR}/${CHANGE}.sh" "${PREFIX}/etc/conda/${CHANGE}.d/${PKG_NAME}_${CHANGE}.sh"
done

And finally the output of conda build

...
make[1]: Leaving directory '$SRC_DIR/po'
Making install in tests
make[1]: Entering directory '$SRC_DIR/tests'
make  install-am
make[2]: Entering directory '$SRC_DIR/tests'
make[3]: Entering directory '$SRC_DIR/tests'
make[3]: Nothing to be done for 'install-exec-am'.
make[3]: Nothing to be done for 'install-data-am'.
make[3]: Leaving directory '$SRC_DIR/tests'
make[2]: Leaving directory '$SRC_DIR/tests'
make[1]: Leaving directory '$SRC_DIR/tests'
Making install in tests/fuzz
make[1]: Entering directory '$SRC_DIR/tests/fuzz'
make[2]: Entering directory '$SRC_DIR/tests/fuzz'
make[2]: Nothing to be done for 'install-exec-am'.
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '$SRC_DIR/tests/fuzz'
make[1]: Leaving directory '$SRC_DIR/tests/fuzz'
make[1]: Entering directory '$SRC_DIR'
make[2]: Entering directory '$SRC_DIR'
 /usr/bin/mkdir -p '$PREFIX/lib'
 /bin/sh ./libtool   --mode=install /usr/bin/install -c   libcryptsetup.la '$PREFIX/lib'
libtool: install: /usr/bin/install -c .libs/libcryptsetup.so.12.10.0 $PREFIX/lib/libcryptsetup.so.12.10.0
libtool: install: (cd $PREFIX/lib && { ln -s -f libcryptsetup.so.12.10.0 libcryptsetup.so.12 || { rm -f libcryptsetup.so.12 && ln -s libcryptsetup.so.12.10.0 libcryptsetup.so.12; }; })
libtool: install: (cd $PREFIX/lib && { ln -s -f libcryptsetup.so.12.10.0 libcryptsetup.so || { rm -f libcryptsetup.so && ln -s libcryptsetup.so.12.10.0 libcryptsetup.so; }; })
libtool: install: /usr/bin/install -c .libs/libcryptsetup.lai $PREFIX/lib/libcryptsetup.la
libtool: finish: PATH="$BUILD_PREFIX/bin:$PREFIX/sbin:$PREFIX/bin:/home/geert/miniforge3/condabin:$BUILD_PREFIX/bin:$PREFIX/bin:/home/geert/miniforge3/bin:/home/geert/miniforge3/condabin:/home/geert/.local/bin:/home/geert/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin" ldconfig -n $PREFIX/lib
ldconfig: /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/libasprintf.so.0 is not a symbolic link

ldconfig: /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/libgettextpo.so.0 is not a symbolic link

----------------------------------------------------------------------
Libraries have been installed in:
   $PREFIX/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the 'LD_RUN_PATH' environment variable
     during linking
   - use the '-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
 /usr/bin/mkdir -p '$PREFIX/sbin'
  /bin/sh ./libtool   --mode=install /usr/bin/install -c cryptsetup veritysetup integritysetup '$PREFIX/sbin'
libtool: install: /usr/bin/install -c .libs/cryptsetup $PREFIX/sbin/cryptsetup
libtool: install: /usr/bin/install -c .libs/veritysetup $PREFIX/sbin/veritysetup
libtool: install: /usr/bin/install -c .libs/integritysetup $PREFIX/sbin/integritysetup
/usr/bin/mkdir -p -m 0755 /$PREFIX/lib/cryptsetup
 /usr/bin/mkdir -p '$PREFIX/include'
 /usr/bin/install -c -m 644 lib/libcryptsetup.h '$PREFIX/include'
 /usr/bin/mkdir -p '$PREFIX/share/man/man8'
 /usr/bin/install -c -m 644 man/cryptsetup.8 man/cryptsetup-open.8 man/cryptsetup-close.8 man/cryptsetup-reencrypt.8 man/cryptsetup-status.8 man/cryptsetup-resize.8 man/cryptsetup-refresh.8 man/cryptsetup-luksFormat.8 man/cryptsetup-luksSuspend.8 man/cryptsetup-luksResume.8 man/cryptsetup-luksAddKey.8 man/cryptsetup-luksRemoveKey.8 man/cryptsetup-luksConvertKey.8 man/cryptsetup-luksKillSlot.8 man/cryptsetup-luksChangeKey.8 man/cryptsetup-erase.8 man/cryptsetup-luksUUID.8 man/cryptsetup-isLuks.8 man/cryptsetup-luksDump.8 man/cryptsetup-luksHeaderBackup.8 man/cryptsetup-luksHeaderRestore.8 man/cryptsetup-token.8 man/cryptsetup-convert.8 man/cryptsetup-config.8 man/cryptsetup-tcryptDump.8 man/cryptsetup-bitlkDump.8 man/cryptsetup-fvault2Dump.8 man/cryptsetup-repair.8 man/cryptsetup-benchmark.8 man/veritysetup.8 man/integritysetup.8 man/cryptsetup-create.8 man/cryptsetup-plainOpen.8 man/cryptsetup-luksOpen.8 man/cryptsetup-loopaesOpen.8 man/cryptsetup-tcryptOpen.8 man/cryptsetup-bitlkOpen.8 man/cryptsetup-fvault2Open.8 man/cryptsetup-luksErase.8 '$PREFIX/share/man/man8'
 /usr/bin/mkdir -p '$PREFIX/lib/pkgconfig'
 /usr/bin/install -c -m 644 lib/libcryptsetup.pc '$PREFIX/lib/pkgconfig'
make[2]: Leaving directory '$SRC_DIR'
make[1]: Leaving directory '$SRC_DIR'

Resource usage statistics from building cryptsetup:
   Process count: 10
   CPU time: Sys=0:00:00.2, User=0:00:02.6
   Memory: 113.4M
   Disk usage: 29.8K
   Time elapsed: 0:00:20.1


Packaging cryptsetup
Packaging cryptsetup-2.7.2-h9bf148f_0
number of files: 69
Warning: rpath /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_build_env/lib is outside prefix /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ (removing it)
Warning: rpath /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_build_env/lib is outside prefix /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ (removing it)
Warning: rpath /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_build_env/lib is outside prefix /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ (removing it)
Warning: rpath /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_build_env/lib is outside prefix /home/geert/miniforge3/conda-bld/cryptsetup_1716734236666/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_ (removing it)

CondaMemoryError: The conda process ran out of memory. Increase system memory and/or try again.
2

We have recently merged a fix for high memory usage during the linkage analysis. Which conda-build version are you using?

3

I’m using conda 24.5.0

4

So yes, that version is affected by the memory issue. A new bugfix will be available this week. Track conda-build v24.5.1 by regro-cf-autotick-bot · Pull Request #227 · conda-forge/conda-build-feedstock · GitHub.

1 Like
5

So I found out, this warning message comes from lib/python3.12/site-packages/conda_build/post.py it checks there if rpath in the binary is set to a path outside of host environment.

So apparently there was… so I decided to use this patchelf to change the rpath to the /lib directory in the host environment.
So after make I looped trough the binaries and corrected the rpath.

But still it fails because post.py uses two method to detect the rpath, and when they don’t agree it still fails…

Does anyone know why these two methods, one using patchelf --print-rpath and two get_rpath_raw from os.utils.liefldd.

I’m confused… why get the same information using two methods?

Why is there a difference when I change the rpath using parchelf for these two methods?