* devshell: better definition of PKG_CONFIG_PATH
that manually going through pkg-config of all packages.
Also filter out non-derivation from nativeBuildInputs.
* Add project.projectVariants that apply flake.variants modules
* Move projectOverlays.devhshell to haskellLib.devshellFor
The time taken to run CI is getting very long. Hopefully removing nixpkgs 21.11 and no longer runs full tests for GHC 9.0.2 well make room for GHC 9.4.1.
Uses nixpkgs-unstable for broken out jobs.
There's a typo in the table of supported versions: two of the 21.11 rows should say `nixpkgs-2111` rather than `nixpkgs-2105`. This threw me off for a bit earlier because I skimmed the table and thought 21.11 wasn't supported yet.
The idea is to keep 8.10.4 in CI while 8.10.5 is still neither in
- a (non-nightly) Stackage LTS resolver
- upstream Nixpkgs-unstable
Hopefully it isn't long before 8.10.5 is more standard and we can remove
8.10.4 from the IOHK build.
This commit also updates documentation (which seems to have fallen out
of date), with some changes and additions to hopefully make things
easier on first-time readers.
* Update default nixpkgs to 20.09
* Move more CI to work on 20.09 by default
* IFD level 1
* Use nixpkgs 20.03 emscripten (needed for ghcjs)
* Add missing materialized files
* Add missing materialized files
* Fix for ghc 8.8 build on darwin
* Fix for ghcjs 8.8 build on darwin
* Fix hls stack test eval for ghc810220201118
* ifdLevel 2
* Oops: accidentally turned on tests for experimental GHC
* Revert "Fix hls stack test eval for ghc810220201118"
This reverts commit b405cd7ee8bd62214bd2d0a61e0317673b5bf1b5.
* ifdLevel 3
* Disable GHC 8.6.5 windows cross on nixpkgs 20.09
* Update supported GHC doc with a table
* Add emscripten comment
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
This change adds a `projectCross` attribute to the return value of
the project functions. It is similar to `pkgs.pkgsCross` and makes
it easier to build cross compiled versions of project (or get
a cross compile with `p.projectCross.${platform}.shellFor`).
It turns out the `otherShells` arg we added to `shellFor` is very
similar to the `inputsFrom` argument of `mkShell`. We can use
`inputsFrom` instead having `otherShells` by replacing the use of
`mkDerivation` with `mkShell` in `shellFor`.
Allow for hoogle to be materialized in `shellFor` by using `tools.hoogle` when passed in.
Co-authored-by: Hamish Mackenzie <Hamish.K.Mackenzie@googlemail.com>
People constantly ask about this, clearly without having found the
documentation that we do have about this. So we need to make it more
obvious:
- Simplify the setup documentation to emphasize the Hydra cache (which
is the only working one currently).
- Move the cache troubleshooting section to the top to make it more
obvious.
- Highlight a link to the troubleshooting section of the docs in the
README.
- Emphasize that there is more documentation than is linked in the
README.
- Add a section to the README that explicitly mentions cache issues, and
links to the corresponding sections of the docs.
It causes a lot of issues. To make this work:
* `shellFor` is updated to use `getAllComponents` instead of `.all`.
* `getAllComponents` is updated to work on the package rather than
the package config.
* Tests updated to not use `.library` or `.exes.X` where appropriate.
* Documentation updated.
* Out of date examples removed.
As a bonus `shellFor` now takes a `components` argument that might be
useful for limiting the dependencies of the shell to just the ones
needed for the components you intend to work on.
Changes to the interface of haskell.nix (from the changelog.md file):
* Removed `sources.nixpkgs-default`, use `sources.nixpkgs` instead.
* Removed `./nixpkgs` directory, use `(import ./. {}).sources`
or `./nix/sources.nix` instead.
* Removes V1 interface for details on how to fix old code see:
https://github.com/input-output-hk/haskell.nix/issues/709
* Removed defaultCompilerNixName.
* cabalProject, cabalProject', hackage-project and hackage-package
now require a `compiler-nix-name` argument.
* `haskell-nix.tool` and `.tools` now require a `compiler-nix-name` argument.
New functions `p.tool` and `p.tools` (where p is a project) do not.
Like `shellFor { tools = ... }` they will use the compiler nix name
from the project (including stack projects where it is derived from
the resolver).
* `haskell-nix.alex` and `haskell-nix.happy` have been removed. Use
`p.tool "alex" "3.2.5"` or `shellFor { tools = { alex = "3.2.5"; } }`.
* `haskell-nix.nix-tools` -> `haskell-nix.nix-tools.ghc883` (it includes
the hpack exe now).
* `haskell-nix.cabal-install` ->
`p.tool "cabal" "3.2.0.0"` or `shellFor { tools = { cabal = "3.2.0.0"; } }`
* `haskell-nix.haskellNixRoots` -> `haskell-nix.roots ghc883` or `p.roots`
Other changes:
Adds hpack executable to the nix-tools derivations.
Adds a `cabal-hpack` test to make sure `hpack` works with
`cabalProject`.
Reduces the number of calls to `cabalProject` (particularly when
checking materialization), by giving internal tools a per-compiler
attribute.
Uses happy 1.19.12 when building newer ghc versions.
Updates cabal-install 3.2.0.0 to use the source from github that
is compatible with ghc 8.10.1.
Updates the docs for callCabalProjectToNix.
Adds a license mapping to fix a common warning.
This overlay helps accessing common executable components.
Typically we want to make these available in a nix-shell
created with shellFor. In most cases the package name
will be the same as the executable, but we have a
`toolPackageName` mapping to help when it is not.
```
# To get a single tool:
haskell-nix.tool "cabal" "3.2.0.0"
# This does the same thing as:
(haskell-nix.hackage-package {
name = "cabal-install"
version = "3.2.0.0"
}).components.exes.cabal
# To get an attr set containing multiple tools:
haskell-nix.tools { cabal = "3.2.0.0"; hlint = "2.2.11"; }
# To add tools to a shell:
shellFor { tools = { cabal = "3.2.0.0"; hlint = "2.2.11"; }; }
```
When used in shellFor the tools will be compiled with the same version
of ghc used in the shell (the build ghc in the case of cross
compilation).
Instead of a version string we can use an attr set containing
arguments that will be passed to `cabalProject`.
For instance to specify the ghc used to compile. Use:
```
haskell-nix.tool "cabal" {
version = "3.2.0.0";
ghc = haskell-nix.compiler.ghc883;
}
```
This change replaces `passthru.run` and `collectRunComponents` with
`haskellLib.check` and `$pkg.checks`.
Uses `ln -s` instead of `cp` for DLLs to avoid duplicating them.
Adds features to help debugging eval issues with haskell.nix/release.nix.
The ifdInputLevel can be set to limit the jobs included to control
how much work is done at the eval stage when looking for issues.
Reduce CI times by excluding older ghc versions from haskellNixRoots
(GHC 8.6.3 and 8.6.4)
Sets `preferLocalBuild=false` on `callCabalProjectToNix` and `callStackToNix`
to ensure they are cached on hydra for both Linux and macOS.
Fix `data-dir:` when cleaning components.
`normalizeRelativeDir` adds a slash on to the end of `dataDir`.
Adding another one here results in `//` and files are left out by
mistake.
* cleanSourceHaskell: Add more filters
* cleanSourceHaskell: Add a "name" parameter, split away the filter
* cleanSourceHaskell: Make it compatible with nixpkgs-19.03
* Adjust usage of cleanSourceHaskell in nix-tools
Currently when a dependency of cabal-install or plan-to-nix changes
the cabal new-configure and plan-to-nix have to be rebuilt in order
to recalculate the plan-nix for bootstrap packages alex, happy and
hscolour. We have pinned the inputs such as hackage index-state
so that the plan generated should be the same. Unfortunately it can
still result in a long wait.
Haskell.nix does cache the result via haskellNixRoots when built on
iohk's hydra server. But if you use a different version of nixpkgs
it is unlikely to work. If you have access to a CI that populates a
nix cache you can add haskellNixRoots to your CI build, but if not
you are kind of stuck.
One option to help with this would be to include the plan-to-nix output
in the haskell.nix git repository. But intermediate files in git is
not nice.
Instead this PR adds just the hashes of the output to the derivations
that runs plan-to-nix. Since the inputs are fixed or should have
no affect on the result (GCC version for instance), we should
be able to rely on them remaining constant.
When trying to build haskell packages from local source using nix (
source code you have used `git clone` to retrieve and have modified)
it is often annoying that small changes can trigger a lot of components
to be built unnecessarily. For instance if you change the code for
a test in a package it will often trigger builds of the library and
all its dependencies.
To avoid this problem we could manually set up `cleanSourceWith` calls
in nix to filter out files that are not need, but to do this every
component would be difficult and error prone.
This change automates this process using the information in the
packages `.cabal` file to filter the source of each component
automatically when using `callCabalPlanToNix` (we should be able to
add it to `callStackToNix` too).
## How does it work?
* A new version on `cleanSourceWith` is used that adds a `subDir`
argument. It allows descending into a subdirectory to compose with
cleaning the source.
* A new `--full` argument is passed to `plan-to-nix` by
`callCabalPlanToNix` telling it to output extra information about
the cabal packages in the plan (directories and filenames).
* `callCabalProjectToNix` uses the new `subDir` argument of
`cleanSourceWith` when setting the packageSrc.
* `comp-builder.nix` and `setup-builder.nix` use a new
`cleanCabalComponent` function to further clean the packageSrc
using the new information in the `package` and `component`.
## hpack wildcards
`package.yaml` files can include wild cards and these are not handled
yet (they do not map to anything in the `.cabal` file and so are lost).
They will need to be added manually to the appropriate field
(eg. `packages.xyz.package.dataFiles`)
If your hpack file does not contain wildcards or if you are willing to
add the wildcards entries manually in nix it is still useful to
enable cleaning on hpack.
For example:
```
packages.ghc-toolkit.package.cleanHpack = true;
packages.ghc-toolkit.components.library.extraSrcFiles = [
"genapply/**/**"
"boot-libs/**/**"
"ghc-libdir/**/**"
];
```
- Dev note about nixpkgs pin
- More information about selecting a GHC version for cabal projects
- A little bit more information about cross compiling, but this is still incomplete.
Jean-Baptiste G
Yvan Sraka
Hamish Mackenzie
endgame
Tikhon Jelvis
Hamish Mackenzie
Tom Prince
Teo Camarasu
Sukant Hajra
Jun Matsushita
Michael Peyton Jones
Eric Wolf
Joe Kachmar
Domen Kožar
Rodney Lorrimar