Help with build variants

Package Builders conda-forge
1

Hello conda community,

we’re maintaining the conda-forge package for CADET (a package for simulating chromatography and other physico-chemical processes) and I would like to get some help with build variants. I tried to read the documentation, as well as the knowledge base but some concepts are still not quite clear to me. Mostly, I don’t fully understand when to use a build matrix in conda_build_config.yaml and when to use different outputs in meta.yaml.

BLAS/MKL

Due to licensing considerations, we want to provide CADET using different BLAS implementations. According to the knowledge base, I added the following to my conda_build_config:

blas_impl:
  - mkl
  - generic

And then add the following to my recipe:

requirements:
  build:
    - {{ compiler("c") }}
    - {{ stdlib("c") }}
    - {{ compiler("cxx") }}
    - git
    - cmake
    - make  # [unix]
  host:
    - mkl-devel             # [blas_impl == "mkl"]
    - libcblas * *_mkl      # [blas_impl == "mkl"]
    - libcblas              # [blas_impl != "mkl"]
    - liblapack             # [blas_impl != "mkl"]
    - tbb-devel             # [enable_threading == "true"]

When building locally, I can now see the options for generic or mkl builds. Is this already enough s.t. the different versions will be provided on conda-forge and depending on the user’s configuration, the corresponding build will be downloaded?

E.g. would running

conda install -c conda-forge "libblas=*=*mkl"
conda install -c conda-forge cadet

then install the version build using MKL?

TBB/noTBB

It’s possible to use TBB for running CADET simulations with multiple threads. However, in many situations running multiple simulations embarrassingly parallel (instead of one simulation on multiple threads) is actually preferable. For this purpose, removing TBB entirely from the build removes overhead and considerably speeds up simulations. Consequently, we would like to provide one version of CADET build with TBB and one without.

My first instinct was to add the following to conda_build_config:

enable_threading:
  - "true"
  - "false"

and adapt meta.yaml accordingly:

requirements:
  build:
    - {{ compiler("c") }}
    - {{ stdlib("c") }}
    - {{ compiler("cxx") }}
    - git
    - cmake
    - make  # [unix]
  host:
    - mkl-devel             # [blas_impl == "mkl"]
    - libcblas * *_mkl      # [blas_impl == "mkl"]
    - libcblas              # [blas_impl != "mkl"]
    - liblapack             # [blas_impl != "mkl"]
    - tbb-devel             # [enable_threading == "true"]
  run:
    - tbb-devel             # [enable_threading == "true"]

This also builds fine locally, but then I wonder, how would users be able to select which version to install?

Should I instead create two “outputs”, e.g. one called cadet and one called cadet-tbb?

Sorry for the long post and I’m looking forward to your answers.

Cheers

2

Yes, but better if the two specs are in the same command:

conda install -c conda-forge cadet "libblas=*=*mkl"

Should I instead create two “outputs”, e.g. one called cadet and one called cadet-tbb?

That would be one way, but maybe too complicated. You can also provide different build strings for each outputs so your users can do conda install cadet=*=*tbb* if they want tbb. You should also choose which one would be prioritized (default) in case they just ask for cadet. This can be done with different build numbers (higher build number => prioritized) or by adding track_features to downprioritize the non-default.

If you have a link to the PR where you are trying to do this, then maybe it’s easier to explain via code reviews.