Source code not included in build package

paco948 · September 5, 2024, 2:45pm

I have to build a conda package for sharing via nexus repository.
The “boa build” command seems to work well, and finally I get that tree structure in the “boa build” directory :

boabuild/
├── channeldata.json
├── index.html
├── linux-64
│ ├── myawesomelib-1.0.0-py312h738df08_0.conda
│ ├── current_repodata.json
│ ├── current_repodata.json.bz2
│ ├── index.html
│ ├── repodata_from_packages.json
│ ├── repodata_from_packages.json.bz2
│ ├── repodata.json
│ └── repodata.json.bz2
└── noarch
├── current_repodata.json
├── current_repodata.json.bz2
├── index.html
├── repodata_from_packages.json
├── repodata_from_packages.json.bz2
├── repodata.json
└── repodata.json.bz2

The log from “boa build” command seems ok, except the fact that the lib is called src-0.0.0 (I think src is because the source code is in src directory).

The source code is in SRC_DIR : /home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/work

Question :
What are the files that must be shared for retrieving the library with mamba install and import : if I share only myawesomelib-1.0.0-py312h738df08_0.conda and repodata.json from linux-64, the mamba install works, but all I have locally is :
./miniforge3/pkgs/myawesomelib-51f24d7-py312h738df08_0
├── info
│ ├── about.json
│ ├── files
│ ├── git
│ ├── hash_input.json
│ ├── index.json
│ ├── paths.json
│ ├── recipe
│ │ ├── conda_build_config.yaml
│ │ ├── LICENSE.md
│ │ ├── meta.yaml
│ │ ├── recipe.yaml
│ │ └── recipe.yaml.template
│ └── repodata_record.json
└── lib
└── python3.12
└── site-packages
└── src-0.0.0.dist-info
├── direct_url.json
├── INSTALLER
├── METADATA
├── RECORD
├── REQUESTED
├── top_level.txt
└── WHEEL
and /home/myaccount/miniforge3/envs/myenv/conda-meta/myawesomelib-51f24d7-py312h738df08_0.json

Log from boa build :
_
| |__ ___ __ _
| '_ \ / _ \ / ` |
| |) | () | (| |
|./ _/ _,_|

Loading config files:

Updating build index: ./boabuild

INFO:conda_index.index.convert_cache:Migrate database
INFO:conda_index.index.convert_cache:CONVERT .cache
INFO:conda_index.index.convert_cache:Migrate database
INFO:conda_index.index:noarch cached 0 B from 0 packages at 0 B/second
INFO:conda_index.index:Subdir: noarch Gathering repodata
INFO:conda_index.index:noarch Writing pre-patch repodata
INFO:conda_index.index:noarch Applying patch instructions
INFO:conda_index.index:noarch Writing patched repodata
INFO:conda_index.index:noarch Building current_repodata subset
INFO:conda_index.index:noarch Writing current_repodata subset
INFO:conda_index.index:noarch Writing index HTML
INFO:conda_index.index:Completed noarch
INFO:conda_index.index:Channeldata subdir: noarch
WARNING: No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.26
Recipe validation OK
Found 1 recipe

myawesomelib

Assembling all recipes and variants

      Output: myawesomelib

┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Package ┃ Variant versions ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ target_platform │ linux-64 │
│ python │ 3.12 │
└─────────────────┴──────────────────┘

Initializing mamba solver

Downloading source

Copying /home/myaccount/eclipse-workspace/myawesomelib/src to /home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/work/

Preparing environment for myawesomelib

Finalizing host for myawesomelib
Reloading output folder: ./boabuild
Finalizing run for myawesomelib
Transaction

Prefix: /home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla

Updating specs:

python=3.12
setuptools[version=‘>=61’]
setuptools-scm[version=‘>=6.2.3’]
pip

Package Version Build Channel Size
────────────────────────────────────────────────────────────────────────────
Install:
────────────────────────────────────────────────────────────────────────────

_libgcc_mutex 0.1 conda_forge localhost Cached
ld_impl_linux-64 2.40 hf3520f5_7 localhost Cached
ca-certificates 2024.8.30 hbcca054_0 localhost Cached
libgomp 14.1.0 h77fa898_1 localhost Cached
_openmp_mutex 4.5 2_gnu localhost Cached
libgcc 14.1.0 h77fa898_1 localhost Cached
openssl 3.3.2 hb9d3cd8_0 localhost Cached
libgcc-ng 14.1.0 h69a702a_1 localhost Cached
libzlib 1.3.1 h4ab18f5_1 localhost Cached
libxcrypt 4.4.36 hd590300_1 localhost Cached
libffi 3.4.2 h7f98852_5 localhost Cached
bzip2 1.0.8 h4bc722e_7 localhost Cached
ncurses 6.5 he02047a_1 localhost Cached
libuuid 2.38.1 h0b41bf4_0 localhost Cached
libnsl 2.0.1 hd590300_0 localhost Cached
libexpat 2.6.2 h59595ed_0 localhost Cached
xz 5.2.6 h166bdaf_0 localhost Cached
tk 8.6.13 noxft_h4845f30_101 localhost Cached
libsqlite 3.46.1 hadc24fc_0 localhost Cached
readline 8.2 h8228510_1 localhost Cached
tzdata 2024a h8827d51_1 localhost Cached
python 3.12.5 h2ad013b_0_cpython localhost Cached
wheel 0.44.0 pyhd8ed1ab_0 localhost Cached
setuptools 73.0.1 pyhd8ed1ab_0 localhost Cached
pip 24.2 pyh8b19718_1 localhost Cached
tomli 2.0.1 pyhd8ed1ab_0 localhost Cached
packaging 24.1 pyhd8ed1ab_0 localhost Cached
typing_extensions 4.12.2 pyha770c72_0 localhost Cached
typing-extensions 4.12.2 hd8ed1ab_0 localhost Cached
setuptools-scm 8.1.0 pyhd8ed1ab_0 localhost Cached

Summary:

Install: 30 packages

Total download: 0 B

────────────────────────────────────────────────────────────────────────────

Transaction starting
Linking _libgcc_mutex-0.1-conda_forge
Linking ld_impl_linux-64-2.40-hf3520f5_7
Linking ca-certificates-2024.8.30-hbcca054_0
Linking libgomp-14.1.0-h77fa898_1
Linking _openmp_mutex-4.5-2_gnu
Linking libgcc-14.1.0-h77fa898_1
Linking openssl-3.3.2-hb9d3cd8_0
Linking libgcc-ng-14.1.0-h69a702a_1
Linking libzlib-1.3.1-h4ab18f5_1
Linking libxcrypt-4.4.36-hd590300_1
Linking libffi-3.4.2-h7f98852_5
Linking bzip2-1.0.8-h4bc722e_7
Linking ncurses-6.5-he02047a_1
Linking libuuid-2.38.1-h0b41bf4_0
Linking libnsl-2.0.1-hd590300_0
Linking libexpat-2.6.2-h59595ed_0
Linking xz-5.2.6-h166bdaf_0
Linking tk-8.6.13-noxft_h4845f30_101
Linking libsqlite-3.46.1-hadc24fc_0
Linking readline-8.2-h8228510_1
Linking tzdata-2024a-h8827d51_1
Linking python-3.12.5-h2ad013b_0_cpython
Linking wheel-0.44.0-pyhd8ed1ab_0
Linking setuptools-73.0.1-pyhd8ed1ab_0
Linking pip-24.2-pyh8b19718_1
Linking tomli-2.0.1-pyhd8ed1ab_0
Linking packaging-24.1-pyhd8ed1ab_0
Linking typing_extensions-4.12.2-pyha770c72_0
Linking typing-extensions-4.12.2-hd8ed1ab_0
Linking setuptools-scm-8.1.0-pyhd8ed1ab_0

Transaction finished

To activate this environment, use:

mamba activate /home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla

Or to execute a single command in this environment, use:

mamba run -p /home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla mycommand

Starting build for myawesomelib

source tree in: /home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/work
export
PREFIX=/home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol
d_placehold_placehold_placehold_placehold_placehold_placehold_pla
export BUILD_PREFIX=/home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/_build_env
export SRC_DIR=/home/myaccount/miniforge3/envs/myawesomelib_env/conda-bld/myawesomelib-0_1725545863744/work
Processing $SRC_DIR
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status ‘done’
Building wheels for collected packages: src
Building wheel for src (setup.py): started
Building wheel for src (setup.py): finished with status ‘done’
Created wheel for src: filename=src-0.0.0-py3-none-any.whl size=927 sha256=f931adf5cab46609c9cf86393b37cc3503a8780a6d712d231eea2095a8410a61
Stored in directory: /tmp/pip-ephem-wheel-cache-m0p35z2f/wheels/23/2a/6d/ff9fb4463cfe59631f1dd8372816d5c36d19f23e4bd992d828
Successfully built src
Installing collected packages: src
Successfully installed src-0.0.0

Resource usage statistics from building myawesomelib:
Process count: 2
CPU time: Sys=-, User=0:00:00.0
Memory: 16.0M
Disk usage: 432B
Time elapsed: 0:00:02.0

number of files: 7
Fixing permissions

Adding files for myawesomelib

lib/python3.12/site-packages/src-0.0.0.dist-info/INSTALLER
lib/python3.12/site-packages/src-0.0.0.dist-info/METADATA
lib/python3.12/site-packages/src-0.0.0.dist-info/RECORD
lib/python3.12/site-packages/src-0.0.0.dist-info/REQUESTED
lib/python3.12/site-packages/src-0.0.0.dist-info/WHEEL
lib/python3.12/site-packages/src-0.0.0.dist-info/direct_url.json
lib/python3.12/site-packages/src-0.0.0.dist-info/top_level.txt

INFO:conda_index.index.convert_cache:Migrate database
INFO:conda_index.index.convert_cache:CONVERT .cache
INFO:conda_index.index.convert_cache:Migrate database
INFO:conda_index.index:linux-64 cached 8 KB from 1 packages at 1.3 MB/second
INFO:conda_index.index:noarch cached 0 B from 0 packages at 0 B/second
INFO:conda_index.index:Subdir: linux-64 Gathering repodata
INFO:conda_index.index:linux-64 Writing pre-patch repodata
INFO:conda_index.index:linux-64 Applying patch instructions
INFO:conda_index.index:linux-64 Writing patched repodata
INFO:conda_index.index:linux-64 Building current_repodata subset
INFO:conda_index.index:linux-64 Writing current_repodata subset
INFO:conda_index.index:linux-64 Writing index HTML
INFO:conda_index.index:Completed linux-64
INFO:conda_index.index:Subdir: noarch Gathering repodata
INFO:conda_index.index:noarch Writing pre-patch repodata
INFO:conda_index.index:noarch Applying patch instructions
INFO:conda_index.index:noarch Writing patched repodata
INFO:conda_index.index:noarch Building current_repodata subset
INFO:conda_index.index:noarch Writing current_repodata subset
INFO:conda_index.index:noarch Writing index HTML
INFO:conda_index.index:Completed noarch
INFO:conda_index.index:Channeldata subdir: linux-64
INFO:conda_index.index:Channeldata subdir: noarch
INFO :: Time taken to mark (prefix)
0 replacements in 0 files was 0.01 seconds

                                    Output: myawesomelib 1.0.0 BN: 0                                        
                                              Variant:                                                   
                                               Build:                                                    
                                                                                                         
                               ╷                     ╷              ╷                         ╷

Dependency │ Version requirement │ Selected │ Build │ Channel
══════════════════════════════════╪═════════════════════╪══════════════╪═════════════════════════╪═════════
│ │ │ │
Host │ │ │ │
python │ 3.12.* │ 3.12.5 │ h2ad013b_0_cpython │
setuptools │ >=61 │ 73.0.1 │ pyhd8ed1ab_0 │
setuptools-scm │ >=6.2.3 │ 8.1.0 │ pyhd8ed1ab_0 │
pip │ │ 24.2 │ pyh8b19718_1 │
…

Thank you,
Lionel

wolfv · September 5, 2024, 3:33pm

Hi, it’s hard to debug this without seeing the recipe, unfortunately.

If you want to install from a local channel, you can also just point to that on the filesystem.

Also I would recommend to try out rattler-build GitHub - prefix-dev/rattler-build: rattler-build is a universal package builder for Windows, macOS and Linux which is the successor of boa

paco948 · September 5, 2024, 6:17pm

Here is the recipe.yaml file :

context:
name: ‘{{ environ[“CONDA_PACKAGE_NAME”] }}’
version: ‘{{ environ[“CONDA_PACKAGE_VERSION”] }}’

package:
name: ‘{{ name|lower }}’
version: ‘{{ version }}’

source:
path: …/src

build:
script: ‘{{ PYTHON }} -m pip install --no-deps --no-build-isolation .’
number: 0

requirements:
host:
- python >=3.12
- setuptools >=61
- setuptools-scm >=6.2.3
- pip
run:
- python >=3.12
- bottleneck 1.3.8
- dask 2024.4.1
- flox 0.9.6
- h5netcdf 1.3.0
- netcdf4 1.6.5
- xarray 2024.3.0

about:
summary: ‘bla bla bla’
dev_url: https://github.com/lionelm948/myawesomelib
license: BSD
license_file: LICENSE.md

I’ll try rattler-build tool, but can I use the same recipe.yaml file I use with boa build ?
Thanks

Topic		Replies	Views
Adding mamba-compatible locking to conda Contributors mamba , conda	1	868	March 23, 2023
Infrastructure Resources Infra resources	5	317	November 17, 2022
Resources for Tool and Infrastructure Contributors Contributors resources	0	476	September 22, 2022
A flurry of Conda Ecosystem Releases ❄️❄️❄️ News conda-mirror , conda-lock , conda , mamba , micromamba , grayskull , conda-libmamba-solver , release , conda-constructor	0	812	February 2, 2023
Multi-User mamba installation Users mamba	1	1723	March 16, 2023

Source code not included in build package

Here is the recipe.yaml file :

Related Topics