How/why is `conda env list` finding other people's environments?

Users conda
1

I’ve been setting up universally available conda environments in miniconda. There are located on an NFS-mounted volume where only root has write access, so I’ve been setting them up with accelerated privileges. Earlier today I wanted to double check what I had named an environment, so ran

conda env list

Much to surprise, it not only listed the environments I created, but also environments created by users in their home directories or in a /scratch partition, both of which are on separate filesystems. Further, the users’ home directories are not even necessarily mounted on the machine I’m working as, as we automount these on demand!

Any insights on what is going on here, and in particular how I can restrict these environment searches to just the publicly installed environments?

1 Like
2

conda keeps a list in ~/.conda/environments.txt. I’d check there!

3

That sees like a reasonable hypotesis; however:

root@finglas:~/.conda# cat environments.txt 
/lusr/opt/miniconda/envs/cuda-dev-11.8
/lusr/opt/miniconda
/lusr/opt/miniconda/envs/cuda-dev-12.0
/lusr/opt/miniconda/envs/cuda-dev-12.2
/lusr/opt/miniconda/envs/cuda-dev-11.3
/lusr/opt/miniconda/envs/pycuda
/lusr/opt/miniconda/envs/pytorch-cpu
/lusr/opt/miniconda/envs/pytorch-cuda
/lusr/opt/miniconda/envs/numpy-numba
/lusr/opt/miniconda/envs/cupy-numba
/lusr/opt/miniconda/envs/cs371n-gpu
/lusr/opt/miniconda/envs/tensorflow-cpu
/lusr/opt/miniconda/envs/tensorflow-cuda

But then if I list environments:

root@finglas:~/.conda# conda env list
WARNING conda.core.envs_manager:list_all_known_prefixes(102): Unable to access /u/bzhou/.conda/environments.txt
WARNING conda.core.envs_manager:list_all_known_prefixes(102): Unable to access /u/changan/.conda/environments.txt
WARNING conda.core.envs_manager:list_all_known_prefixes(102): Unable to access /u/nskiran/.conda/environments.txt
(there are a number of these)
base                     /lusr/opt/miniconda
cs371n-cpu               /lusr/opt/miniconda/envs/cs371n-cpu
cs371n-gpu               /lusr/opt/miniconda/envs/cs371n-gpu
cuda-dev-11.3            /lusr/opt/miniconda/envs/cuda-dev-11.3
cuda-dev-11.8            /lusr/opt/miniconda/envs/cuda-dev-11.8
cuda-dev-12.0            /lusr/opt/miniconda/envs/cuda-dev-12.0
cuda-dev-12.2            /lusr/opt/miniconda/envs/cuda-dev-12.2
cupy-numba               /lusr/opt/miniconda/envs/cupy-numba
numpy-numba              /lusr/opt/miniconda/envs/numpy-numba
pycuda                   /lusr/opt/miniconda/envs/pycuda
pytorch-cpu              /lusr/opt/miniconda/envs/pytorch-cpu
pytorch-cuda             /lusr/opt/miniconda/envs/pytorch-cuda
tensorflow-cpu           /lusr/opt/miniconda/envs/tensorflow-cpu
tensorflow-cuda          /lusr/opt/miniconda/envs/tensorflow-cuda
                     /lusr/opt/miniconda/user-envs/pgoetz/torch
                     /lusr/opt/miniconda/user-envs/pgoetz/torch2
                     /scratch/cluster/abhinavc/miniforge3
                     /scratch/cluster/abhinavc/miniforge3/envs/d2
                     /scratch/cluster/abhinavc/miniforge3/envs/d2_parseq
                     /scratch/cluster/abhinavc/miniforge3/envs/parseq
                     /scratch/cluster/eladlieb/miniconda3
                     /scratch/cluster/eladlieb/miniconda3/envs/adamarl
                     /scratch/cluster/eladlieb/miniconda3/envs/nocturne
                     /scratch/cluster/eladlieb/miniconda3/envs/updet
                     /scratch/cluster/harwath/packages/anaconda3
                     /scratch/cluster/harwath/packages/anaconda3/envs/rdvq
                     /scratch/cluster/patmac/anaconda3/envs/3dsim
                     /scratch/cluster/sanmit/conda
                     /scratch/cluster/sanmit/conda/envs/babyai
                     /scratch/cluster/sanmit/conda/envs/l2m
                     /scratch/cluster/sanmit/conda/envs/rlpyt-babyai
                     /scratch/cluster/sanmit/conda/envs/tf-babyai
                     /scratch/cluster/sanmit/conda/envs/tfbabyai
                     /scratch/cluster/soroush/software/miniconda3/envs/rpl
                     /scratch/cluster/stephane/miniconda3
                     /scratch/cluster/stephane/miniconda3/envs/robosuite
                     /scratch/cluster/theksong/miniconda3
                     /scratch/cluster/theksong/miniconda3/envs/anerf
                     /scratch/cluster/william/anaconda3/envs/memory
                     /scratch/cluster/yifan/anaconda3
                     /u/anton/miniconda3
                     /u/anton/miniconda3/envs/test
                     /u/ecprice/miniconda3
                     /u/ecprice/miniconda3/envs/env
                     /u/mehnaz/anaconda3
                     /u/mrudolph/miniconda3
                     /u/vl/miniconda3

The entries in /u are automounted directories which might or might not be visible from a machine. This makes it more confusing how it’s finding all these, as I"m certain most of these users have never logged in to this host.

4

Well, turns out that function does a bit too much :joy:

1 Like