* Wed Jul 10 2024 Egbert Eich <eich@suse.com>
- Update to 0.22.1.
* Bug Fixes:
- Fix reuse of externals on Linux.
- Ensure parent gcc-runtime version >= child.
- Ensure the latest gcc-runtime is rpath'ed when multiple exist
among link deps.
- Improve version detection of glibc.
- Improve heuristics for solver.
- Make strong preferences override reuse.
- Reduce verbosity when C compiler is missing.
- Make missing ccache executable an error when required.
- Make every environment view containing `python` a `venv`.
- Fix external detection for compilers with os but no target.
- Fix version optimization for roots.
- Handle common implementations of pagination of tags in OCI
build caches.
- Apply fetched patches to develop specs.
- Avoid Windows wrappers for filesystem utilities on non-Windows.
- Fix formatting issue in `spack audit`.
* Package updates:
- Require libiconv for iconv.
Notice that glibc/musl also provide iconv, but are not
guaranteed to be complete. Set `packages:iconv:require:[glibc]`
to restore the old behavior.
- protobuf: fix 3.4:3.21 patch checksum.
- protobuf: update hash for patch needed when="@3.4:3.21".
- git: bump v2.39 to 2.45; deprecate unsafe versions.
- gcc: use `-rpath {rpath_dir}` not `-rpath={rpath dir}`.
- Remove mesa18 and libosmesa.
- Enforce consistency of `gl` providers.
- py-matplotlib: qualify when to do a post install.
- rust: fix v1.78.0 instructions.
- suite-sparse: improve setting of the libs property.
- netlib-lapack: provide blas and lapack together.
* Thu May 16 2024 Egbert Eich <eich@suse.com>
- Move-site-config-scope-before-system-scope.patch:
Give 'site' scope a lower precedence than 'system' scope.
The 'site wide' config scope was meant to be per Spack
installation. A single system may have multiple Spack
installations, so was is meant for overriding the
'system' wide setting per installation.
The Spack package is OS-vendor provided. The vendor provides
pr generates a configuration which a local admin may want to
override. This can now be done from within the 'system' scope.
Previously the vendor-supplied configuration was mixed with
the 'system' scope - local modifications collided with
vendor autoconfiguration.
- Add a build-dependency package which will cause build tools
and libraries used frequently by Spack to be installed.
All these packages are recommended by the main Spack package
already. This package may be used in environments where the
installation of recommended packages is disabled by default.
* Mon May 13 2024 Egbert Eich <eich@suse.com>
- Update Spack to version 0.22.0
* New features:
- Compiler dependencies:
Spack is in the process of making compilers proper dependencies.
For this, compiler dependencies are moving from `compilers.yaml`
to `packages.yaml` to make this consistent with other externals.
For this, dependency graphs will not show the compiler runtime
libraries like `gcc-runtime` or `libgfortran`.
To minimize disruption, an existing `compilers.yaml` file will
continue to work, however, users are encourage to migrate
before v0.23.
+ Packages compiled with `%gcc` now depend on a new package
`gcc-runtime`, which contains a copy of the shared compiler
runtime libraries. This enables gcc runtime libraries to be
installed and relocated when using a build cache.
When building minimal Spack-generated container images it
is no longer necessary to install libgfortran, libgomp
etc. using the system package manager.
+ Packages compiled with `%oneapi` now depend on a new package
`intel-oneapi-runtime`. This is similar to gcc-runtime.
This allows us to model library soname compatibility and
allows compilers like %oneapi to provide virtuals like sycl
(which can also be provided by standalone libraries).
+ Changes to the optimization criteria of the solver improve
the hit-rate of buildcaches by a fair amount. The solver has
more relaxed compatibility rules and will not try to strictly
match compilers or targets of reused specs. Users can still
enforce the previous strict behavior with require: sections
in packages.yaml.
Note that to enforce correct linking, Spack will not reuse
old %gcc and %oneapi specs that do not have the runtime
libraries as a dependency.
+ Spack will reuse specs built with compilers that are not
explicitly configured in `compilers.yaml`. Because we can
now keep runtime libraries in build cache, we do not require
you to also have a local configured compiler to use the
runtime libraries. This improves reuse in buildcaches and
avoids conflicts with OS updates that happen underneath
Spack.
+ Binary compatibility on `linux` is now based on the `libc`
version, instead of on the OS tag. Spack builds now detect
the host `libc` (`glibc`) and add it as an implicit external
node in the dependency graph. Binaries with a `libc` with
the same name and a version less than or equal to that of
the detected libc can be reused.
+ Each package that can provide a compiler is now detectable
using spack external find. External packages defining
compiler paths are effectively used as compilers, and spack
external find -t compiler` can be used as a substitute for
`spack compiler find. More details on this transition are
in [the docs](https://spack.readthedocs.io/en/latest/getting_started.html#manual-compiler-configuration).
- Improved spack find UI for Environments:
`spack find` in environments shows information about
* What are the roots
* Which ones are installed / not installed
* What's been added that still needs to be concretized
much more clearly. An option `--only-roots`/`-r` has been added
to `spack find` that will only show env roots, if you don't want
to look at all the installed specs.
- Improved command-line string quoting
Spack now parses specs in the CLI respecting shell quoting. Instead
of:
```
spack install zlib cflags=\"-O2 -g\"
```
you should use:
```
spack install zlib cflags="-O2 -g"
```
The old form will now result in an error.
This allows to include special characters, flags like
```
spack intall zlib ldflags='-Wl,-rpath=$ORIGIN/_libs'
```
may now be supplied. To reduce ambiguities during paring, quotes
are no longer allowed around `=` and `==`. Commands like:
```
spack install zlib cflags = "-O2 -g"
```
will result in an error.
- Revert default spack install behavior to `--reuse`
In v0.20 the default concretizer behavior was changed from
`--reuse` to `--reuse-deps`, which meant that *every* `spack
install` invocation would attempt to build a new version of
the requested package / any environment roots.
While this is a common ask for *upgrading* and for *developer
workflows*, it shouldn't be the default for a package manager.
- The `install` command now offers the three options:
* `--reuse` (default): reuse as many existing installations
as possible.
* `--reuse-deps` or `--fresh-roots`: upgrade (freshen) roots
but reuse dependencies if possible.
* --fresh: install fresh versions of requested packages
(roots) and their dependencies.
- More control over reused specs
Spack now provides better control over which packages to reuse
and how. There is a new `concretizer:reuse` config option,
which accepts the following properties:
* `roots`: true to reuse roots, false to reuse just dependencies
* `exclude`: list of constraints used to select which specs
not to reuse.
* `include`: list of constraints used to select which specs
to reuse.
* `from`: list of sources for reused specs (some combination
of `local`, `buildcache`, or `external`).
- New `conflict:` and `prefer:` syntax for package preferences
Instead of having to express conflicts and preferences just
using `require:`, these can now be expressed explicitly.
- `include_concrete` in environments
This directive allows to build on the *cncrete* contents of
another environment without changing this environment by
includeing the concrete specs from this environment's
`spack.lock`:
```
spack:
specs: []
concretizer:
unify: true
include_concrete:
- /path/to/environment1
- /path/to/environment2
```
When this environment is concretized, it will bring the
already concrete specs from `environment1` and `environment2`,
and build on top of them without changing them.
- `python-venv` isolation
Spack now insert a small `python-venv` package in between
`python` and packages that need to install Python code to
isolate Spack's build environment and shield Spack form
potential issues with an external python.
- Packages can now specify whether they may be distributed in
source or binary form.
* New commands, options and directives
- Allow packages to be pushed to build cache after install from
source.
_ `spack develop`: stage build artifacts in same root as non-dev
builds.
- Don't delete spack develop build artifacts after install.
- `spack find`: add options for local/upstream only.
- `spack logs`: print log files for packages (either partially
built or installed).
- `patch`: support reversing patches.
- `develop`: Add `-b`/`--build-directory` option to set
build_directory package attribute.
- `spack list`: add `--namesapce` / `--repo` option.
- spack gc: add options for environments and build dependencies.
- Add `--create` to `spack env activate`.
* Performance improvements
- environment.py: Fix excessive re-reads.
- ruamel yaml: Fix quadratic complexity bug.
- Refactor to improve `spec format` speed.
- Do not acquire a write lock on the env post install if no views.
- `asp.py`: fewer calls to `spec.copy()`
- `spec.py`: early return in `__str__`
- avoid `jinja2` import at startup unless needed.
* Other new features of note
- `archspec`: update to v0.2.4: bugfixes for `neoverse-v1` and
`neoverse-v2` detection.
- `spack config` get/blame: with no args, show entire config
- `spack env create <env>`: dir if dir-like
- Add handling of custom ssl certs in urllib ops.
- Add ability to rename environments.
- Add config option and compiler support to reuse across OS's.
- Support for prereleases.
- Only reuse externals when configured.
- Environments: Add support for including views.
* Binary caches
Build cache: make signed/unsigned a mirror property.
* Removals, deprecations, and syntax changes
- Remove `dpcpp` compiler and package.
- `spack load`: remove `--only` argument.
* Notable Bugfixes
- repo.py: drop deleted packages from provider cache.
- Allow `+` in module file names.
- `cmd/python`: use runpy to allow multiprocessing in scripts.
- Show extension commands with `spack -h`.
- Support environment variable expansion inside module
projections.
- Alert user to failed concretizations.
- `shell`: fix `zsh` color formatting for PS1 in environments.
- `spack mirror create --all`: include patches.
- Update `run-find-external.sh` script to address the deprecation
of `compilers.yaml`.
* Thu Apr 18 2024 Egbert Eich <eich@suse.com>
- Add lsb-release and zstd as new requires to match upstream
prerequisites.
* Thu Apr 04 2024 Egbert Eich <eich@suse.com>
- Make `patchelf` a mandatory dependency on products where it
is always available.
* Fri Mar 15 2024 Egbert Eich <eich@suse.com>
- Update Spack to version 0.21.2
* Bugfixes
+ Containerize: accommodate nested or pre-existing `spack-env`
paths.
+ Fix `setup-env` script, when going back and forth between
instances.
+ Fix using fully-qualified namespaces from root specs.
+ Fix a bug when a required provider is requested for multiple
virtuals.
+ OCI buildcaches:
* only push in parallel when forking.
* use pickleable errors (#42160)
+ Fix using sticky variants in externals.
+ Fix a rare issue with conditional requirements and
multi-valued variants.
* Package updates
+ `rust`: add v1.75, rework a few variants.
+ `py-transformers`: add v4.35.2.
- Fix path to setup-env.sh in the Apptainer template (bsc#1221471).
- Add libgfortran, libfl2 and libzip5 to the Spack runtime
container as the Spack build container has the corresponding
devel packages but these libraries are not installed in a
BCI-style base container by default (bsc#1221503).
* Fri Feb 02 2024 Egbert Eich <eich@suse.com>
- Make python version used configurable.
- Filter out more rpmlint errors.
* Thu Jan 25 2024 Christian Goll <cgoll@suse.com>
- Update Spack to version 0.21.1
* Add support for reading buildcaches created by Spack v0.22
* Bugfixes
+ `spack graph`: fix coloring with environments
+ `spack info`: sort variants in --variants-by-name
+ `Spec.format`: error on old style format strings
+ ASP-based solver:
- fix infinite recursion when computing concretization
errors,
- don't error for type mismatch on preferences,
- don't emit spurious debug output.
+ Improve the error message for deprecated preferences.
+ Fix multi-word aliases.
+ Add a warning for unconfigured compiler.
+ environment: fix an issue with deconcretization/reconcretization
of specs.
+ buildcache: don't error if a patch is missing, when
installing from binaries
- From version 0.21.0
* following new features:
+ Better error messages with condition chaining:
In v0.18, we added better error messages that could tell you
what problem happened, but they couldn't tell you why it
happened. 0.21 adds condition chaining to the solver, and
Spack can now trace back through the conditions that led to
an error and build a tree of causes potential causes and
where they came from.
+ OCI build caches:
You can now use an arbitrary OCI registry as a build cache:
- For Dockerhub:
`$ spack mirror add my_registry oci://user/image`
- For another registry (GHCR):
`$ spack mirror add my_registry oci://ghcr.io/haampie/spack-test`
Then set the login credentials:
`$ spack mirror set --push --oci-username ... --oci-password ... my_registry`
and push to it:
`$ spack buildcache push my_registry [specs...]`
You can optionally add a base image to get runnable images:
```
$ spack buildcache push --base-image leap:15.5 my_registry python`
Pushed ... as [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack
$ docker run --rm -it [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack
```
This creates a container image from the Spack installations
on the host system, without the need to run `spack install`
from a `Dockerfile` or `sif` file. It also addresses the
inconvenience of losing binaries of dependencies when
`RUN spack install` fails inside `docker build`. Further, the
container image layers and build cache tarballs are the same
files. This means that `spack install` and `docker pull` use the
exact same underlying binaries. If you previously used `spack
install` inside of docker build, this feature helps you save
storage by a factor two.
+ Multiple versions of build dependencies:
Increasingly, complex package builds require multiple
versions of some build dependencies. For example, Python
packages frequently require very specific versions of
`setuptools`, `cython`, while different physics packages
require different versions of Python to build. The concretizer
enforced that every solve was unified, i.e., so that there was
only one version of every package. The concretizer now supports
"duplicate" nodes for build dependencies, but enforces unification
through transitive link and run dependencies. This will allow it
to better resolve complex dependency graphs in ecosystems like
Python.
+ Cherry-picking virtual dependencies:
You can now select only a subset of virtual dependencies
from a spec that may provide more. For example, to make mpich
your mpi provider, you can be explicit by writing:
`hdf5 ^[virtuals=mpi] mpich`.
Or, to use, e.g., `intel-parallel-studio` for blas along with
an external `lapack` like `openblas`, you could write:
```
strumpack ^[virtuals=mpi] intel-parallel-studio+mkl ^[virtuals=lapack] openblas`
```
The `virtuals=mpi` is an edge attribute, and dependency edges
in Spack graphs now track which virtuals they satisfied.
+ The `spack deconcretize` command gives you control over what
you want to update in an already concrete environment.
As an example, with an environment built with meson, and you
want to update your meson version, you can run:
`$spack deconcretize meson`
and have everything that depends on meson rebuilt the next
time you run spack concretize. In the future, we'll handle
this in a single command, but for now you can use this to
drop bits of your lockfile and resolve your dependencies
again.
+ UI Improvements:
The `spack info` received a rework to make the output more
appealing. It is now on par with the rest of Spack's UI.
`spack info` now makes much better use of terminal space and
shows variants, their values, and their descriptions more
clearly. Conditional variants are grouped separately so you
can more easily understand how packages are structured.
`spack checksum` now allows you to filter versions from your
editor, or by version range. It also notifies you about
potential download URL changes.
+ Environments can include definitions:
Spack did not previously support using `include:` with The
definitions section of an environment, but now it does. You
can use this to curate lists of specs and more easily reuse
them across environments.
+ Aliases:
You can now add aliases to Spack commands in `config.yaml`,
e.g. this might enshrine your favorite args to `spack find`
as `spack f`:
```
config:
aliases:
f: find -lv
```
+ Improved autoloading of modules:
In this release, you can start using `hide_implicits: true`
instead, which exposes only explicitly installed packages to
the user, while still autoloading dependencies. On top of
that, you can safely use `hash_length: 0`, as this config now
only applies to the modules exposed to the user -- you don't
have to worry about file name clashes for hidden
dependencies.
Note: for Tcl this feature requires Modules 4.7 or higher.
* Other new commands and directives:
+ `spack env activate` without arguments now loads a default
environment that you do not have to create.
+ `spack find -H` / `--hashes`: a new shortcut for piping spack
find output to other commands.
+ Add `spack checksum --verify`, fix `--add`.
+ New `default_args` context manager factors out common args for
directives.
+ `spack compiler find --[no]-mixed-toolchain` lets you easily
mix clang and gfortran on Linux.
* Performance improvements:
+ `spack external find execution` is now much faster.
+ `spack location -i` is now much faster on success.
+ Drop redundant rpaths post install.
+ ASP-based solver: avoid cycles in clingo using hidden
directive.
+ Fix multiple quadratic complexity issues in environments.
* Other new features of note:
+ archspec: update to v0.2.2, support for Sapphire Rapids,
Power10, Neoverse V2.
+ Propagate variants across nodes that don't have that variant
+ Implement fish shell completion.
+ Can now distinguish between source/binary mirror; don't ping
mirror.spack.io as much.
+ Improve status reporting on `spack install`
(add [n/total] display...).
* Mon Nov 06 2023 Bernhard Wiedemann <bwiedemann@suse.com>
- Update to version 0.20.3 with the following changes:
* Bug fixes:
+ Fix a bug where `spack mirror set-url` would drop configured
connection info.
+ Fix a minor issue with package hash computation for Python 3.12.
+ Improve escaping in Tcl module files.
+ Make repo cache work on repositories with zero mtime.
+ Ignore errors for newer, incompatible buildcache version.
+ Print an error when git is required, but missing.
+ Ensure missing build dependencies get installed when using
`spack install --overwrite`.
+ Fix an issue where Spack freezes when the build process
unexpectedly exits.
+ Fix a bug where installation failures cause an unrelated
`NameError` to be thrown.
+ Fix an issue where Spack package versions would be incorrectly
derived from git tags.
+ Fix a bug triggered when file locking fails internally.
+ Prevent `spack external find` to error out when a directory
cannot be accessed.
+ Fix multiple performance regressions in environments.
+ Add more ignored modules to `pyproject.toml` for `mypy`.
* Features:
+ Spack now supports Python 3.12.
* Tue Sep 12 2023 Egbert Eich <eich@suse.com>
- Update to version 0.20.1 with the following changes:
* Bug fixes:
+ Fix spec removed from an environment where not actually
removed if `--force` was not given.
+ Hotfix for a few recipes that treat CMake as a link
dependency.
+ Fix re-running stand-alone test a second time, which was
getting a trailing spurious failure.
+ Fix reading JSON manifest on Cray, reporting non-concrete
specs.
+ Fix a few bugs when generating Dockerfiles from Spack.
+ Fix a few long-standing bugs when generating module files.
+ Fix issues with building Python extensions when using an
external Python.
+ Fix `spack compiler remove`: remove from command line even
if they appear in different scopes.
* Features:
+ Speed-up module file generation.
+ Show external status as `[e]`.
+ Backport `archspec` fixes.
+ Improve a few error messages.