Conda channel preferences

Users How do I ... ?
1

Hi,

Recently I’ve started to use conda as my main environment management tool, and have been using it using the default settings (and channels). Recently, I’ve been investigating PyPy which required me to start caring about which conda channels I use as PyPy mandates the use of the conda-forge channel

Off the back of this experience, I’m left with some questions:

  • When using conda, do I need to commit to using conda only with specific channels (specifically - conda-forge)?
    I ask this because when I was installing a package from conda-forge, it was prompting me to update my conda installation which confused me a little as I had only just updated it (though of course from default channels). Attempting to update using conda update -n base -c conda-forge conda yielded a vast array of updates (basically everything looked like it required updating!). I stopped short of accepting the updates, as it occurred to me that if I did this, it may break everything which I wished to use outside of conda-forge (eg every pre-existing environment that I created using default channels)

  • As a corollary to the above, if I update conda from a specific channel, am I committed to solely using that channel for all future environment creation?
    One would imagine this is the case. Otherwise, why are conda updates from one channel so spectacularly different from those on default channels? Presumably (and this is based from recent experience), one cannot “mix and match” packages from different channels

  • If I can’t “mix and match” packages from different channels, so require environments using different channels, do I need to set up multiple conda environments to target separate channels?

Appreciate any advice the community could provide!

Thanks.

2

It is perfectly fine to use a single conda installation to manage some environments that use defaults and some that use conda-forge. However, it’s safest not to mix defaults and conda-forge within the same environment.

If you used vanilla conda (i.e., not something like miniforge that is set up to use conda-forge from the get-go), it’s best not to mix conda-forge into your base environment.

In general, actually, it’s best not to do much of anything in the base environment. Keep the base environment just for conda itself, using the defaults channel, and don’t install other stuff there. Then you can create different environments to use for different purposes, and you can set some of those environments to use conda-forge and some to use defaults, and all will be well.

One confusing thing is that specifying -c conda-forge when creating the environment[1] does not actually set conda-forge as a persistent channel for the environment. It just tells conda to search conda-forge for those particular packages you’re including when you create the env, but if you later do conda install blah in that env, it will still look in defaults, not conda-forge. To fix this, after creating and activating the environment, you can do conda config --env --add channels conda-forge to set conda-forge as the preferred channel for future installs within that env.

With regard to your more conceptual questions: The main issues arise from mixing channels that provide the same packages. In some situations, defaults and conda-forge can wind up acting as separate dependency trees. So suppose you install package X which depends on A, B and C. If you install package X from defaults, it will want A, B, and C from defaults; if you install X from conda-forge it will want A, B, and C also from conda-forge. This is why you get the “vast array of updates” that you saw. Because of that, as you surmised, it’s best not to mix conda-forge and defaults within the same environment if you can avoid it, as it can lead to conda wanting to bounce back and forth between the two channels every time you change something.

It is possible to mix channels, and it can be okay in cases where one channel is “small” and doesn’t provide the same packages as the “main” channel (defaults or conda-forge) for that env. For instance, sometimes you’ll encounter a situation where an individual has made conda builds of a package available on their own anaconda channel. Then you can sometimes do conda install -c somebodys_channel that_persons_package to install a particular package from their channel and (assuming all the dependencies are specified right) it will likely work. The problems usually come, as I mentioned, when multiple active channels provide the same package and conda gets confused about which one to use.

  1. e.g., conda create -n myenv -c conda-forge python=3.11 ↩︎

2 Likes
3

Awesome response Brendan…thanks for clarifying those points. I’m using miniconda as opposed to miniforge. Accordingly…and in line with your advice and my instincts…I’ll leave the base environment well alone apart from updating from the default environments.

Again…thanks for taking the time to create such a complete and thoughtful response :+1:

1 Like