conda-package-streaming 0.8.0 and conda-package-handling 2.1.0 have been released. These releases are mostly to add the extracted size to the
.tar.zst archives that form part of a
.conda file, allowing
zstd to allocate less memory on decompression when the extracted size is less than the default buffer size for that compression level (most noticeable when using compression levels >= 20).
To accomplish this
conda-package-handling iterates over its input files twice when creating a
.conda from scratch when
conda-build is configured to emit
.conda directly, and
conda-package-streaming saves uncompressed
.tar to a temporary file when doing
transmute (also powers the
cph transmute command). We expect most users will not notice as the majority of time is spent inside the
zstd compressor but
transmute will require more disk space.
This will also be the first
conda-package-handling release to be on
pypi as soon as we are able to claim its package name there.
Changelogs for the releases.
conda-package-handling 2.1.0 (2023-05-04)
- Include decompressed size when creating
CondaFormat_v2.create(), to reduce memory usage on decompression. (#171) Transmuted archives (converted from
.tar.bz2) do not contain the decompressed size. NOTE: Unless using conda-package-streaming >= 0.8.0
- Include LICENSE, not just LICENSE.txt in info/ section (#172)
conda-package-streaming 0.8.0 (2023-05)
- Update transmute to use SpooledTemporaryFile instead of streaming directly to
zip (#57). This
can reduce zstd memory usage during decompression.
transmutereturns Path to transmuted package instead of