* Add `package.buildable` option
Turning off building for a whole package requires setting `buildable = lib.mkForce false` on each of the components of the package. This change adds support for `package.buildable = false` to make it easier.
So:
```
packages.marlowe-actus.components.library.buildable = lib.mkForce false;
packages.marlowe-actus.components.exes.marlowe-actus-test-kit.buildable = lib.mkForce false;
packages.marlowe-actus.components.tests.marlowe-actus-test.buildable = lib.mkForce false;
```
Can be replaced with:
```
packages.marlowe-actus.package.buildable = false;
```
This change updates to the latest `nix-tools` to get the following fixes (there are 3 PRs in nix-tools, but just the one in haskell.nix to avoid having to update the materialized files multiple times):
## Better support for source repository packages
* https://github.com/input-output-hk/nix-tools/pull/107
Currently these are replaced by the `cabalProject` functions with regular `packages:` before running cabal configure. Cabal does not treat these the same (the setting of `tests:` and `benchmarks:` in the `cabal.project` file):
* The plan found by `cabalProject` may not match the one used when running `cabal`.
* The performance of the solver may not be consistent with running `cabal`.
This change replaces `source-repository-package` with another `source-repository-package` pointing at a minimal git repo.
## Only include planned components
* https://github.com/input-output-hk/nix-tools/pull/108
Only the components in the `plan.json` are now included in the haskell.nix cabal projects. This avoids missing dependencies attempting to build components that were not in the plan. Should fix#993.
## Pick latest packages
* https://github.com/input-output-hk/nix-tools/pull/109
When the same package occurs more than once in a `plan.json` file (perhaps because it is needed both by the project itself and by one of the `setup` dependencies or `build-tool-dependencies` of the project) the latest version will now be the one picked by haskell.nix. This is a work around for a common issue with `cabal-doctest` when cross compiling to windows (an old version of Win32 is used even if a newer one was required by the projects `constraints`).
The `hix` tools are wrappers for the various `nix` tools that
use `haskell.nix` without the need to add any `.nix` files.
This is useful for:
* A quick way to try out haskell.nix for new users.
* Using haskell.nix to work on projects that do not have
`.nix` files.
* Testing to see if `haskell.nix` can build a project.
* Making `flake` and `non flake` configurations to check `haskell.nix`
treats them the same.
The configuration arguments for `Hix` can be (from highest precedence to lowest):
* Passed on the command line with `--arg` (or `--argstr` for string args).
* Placed in `nix/hix.nix` file in the project dir.
* Placed in `~/.config/hix/hix.conf`
Boilerplate `default.nix`, `shell.nix` and `flake.nix` files can be added to a
a project with a `nix/hix.nix` file to make it work with the standard `Nix`
tools.
There are now 4 project modules used to check the arguments passed to the various project functions:
* `project-common.nix` - Arguments used by all the project functions
* `stack-project.nix` - Used by the `stackProject` and `stackProject'` functions
* `cabal-project.nix` - Used by the `cabalProject` and `cabalProject'` functions
* `project.nix` - Just the `projectFileName` argument that is used by `project` and `project'` functions to determine whether to call `stackProject` or `cabalProject` function.
This also includes the `rawProject.args` that was mistakenly left out of #1141 causing #1142 and improvements for the docs for the use of the `shell` argument in `flake.nix` files.
* Bump nix-tools to latest master revision
* Add writeHieFiles
* Update modules/plan.nix
Co-authored-by: Michael Peyton Jones <me@michaelpj.com>
* Move where the hie flags are specified
Co-authored-by: Michael Peyton Jones <me@michaelpj.com>
The default shell `devShell` works by adding a default `shell` to all projects. You can pass the shell arguments to the project function using the new `shell` argument:
```
let myProject = haskell-nix.project {
src = ./.;
shell.tools = { cabal = {}; };
shell.crossPlatforms = p: [ p.cross ];
};
```
This will include js-unknown-ghcjs-cabal in the `devShell`.
To add cross compiled outputs to the flake pass `crossPlatforms` to the `flake` function.
```
myProject.flake { crossPlatforms = p: [ p.cross ]; }
```
To cross compile a component include the platforms `config` string to the output name like this
```
nix build .#js-unknown-ghcjs:pkg-name:lib:pkg-name
```
* Use nix module system to check project arguments
This change uses the nix module system for the arguments to:
* `cabalProject'`
* `cabalProject`
* `stackProject'`
* `stackProject`
This will:
* Give an error when the name or type of an argument is incorrect
(currently misnamed args are quietly ignored).
* Fixes a problem with `projectCross` not having a way to pass different args to `cabalProject`
for different platforms. We can now use:
`({pkgs, ...}: { cabalProjectLocal = if pkgs.stdenv.hostPlatform.isX then ... })`
* Add test.
* wip
* Fix merge regression
* Combine drv, drv.source and drv.dist for doctest
* Skip cabal-doctests test when cross compiling
* Add the --no-magic flag to the .cabal file for the cabal-doctests test.
This appears to be necessary on OSX.
The --no-magic flag stops the doctest executable from expanding path
arguments, trying to locate the package db, etc.
This shouldn't be necessary with cabal-doctest, since all the
necessary options to pass to doctest are computed when running the Setup.hs
script.
See
https://github.com/input-output-hk/haskell.nix/pull/427#issuecomment-605761780.
* Fix cabal-doctest support
* Skip cabal-doctest test plan cross compiling
* More fixes for cabal-doctest
* Skip cabal-doctest tests when cross compiling
Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
Co-authored-by: Hamish Mackenzie <Hamish.K.Mackenzie@gmail.com>
`ghcOptions` has been moved from package and is now a list of strings.
old: packages.x.package.ghcOptions = "someGHCoption";
new: packages.x.ghcOptions = ["someGHCoption"];
To specify ghcOptions for all packages:
ghcOptions = ["someGHCoption"];
For a single component:
packages.x.compoents.library.ghcOptions = ["someGHCoption"];
See https://github.com/input-output-hk/nix-tools/pull/101
We should look up hsPkgs.${pkg-name}.components.exe.${component-name}
instead of including the package as a dependency.
Adding `hsPkgs.${pkg-name}` a tool it is not clear which executable in the package `haskell.nix` should choose.
Haskell.nix did not deal with this well and in fact it could lead to `nix-shell` crashing. For instance using `shellFor` to make a shell for building `haskell-language-server` with `cabal build` crashed as a dependency on `ghcide` the executable (by the `ghcide-bench` executable) caused infinite recursion. Allowing `ghcide-bench` to correctly depend just on `components.exes.ghcide` fixes this.
This PR also includes:
* Updated materialized files built with the new nix-tools and latest index-state
* Small update to GHCJS (pins happy) to fix issue that showed up when the materialization was updated
* A fix for the infinite recursion issue when updating materialized files (by adding ghc-boot-packages-unchecked).
* Performance fix for shellFor (checks just the names of component derivations when filtering).
I think this is more correct: other package that depend on the system
odbc package may not need freetds, the problem is that the Haskell odbc
package doesn't declare its dependency on freetds.
* Add parser combinators to parse SPDX license expressions
The most important parser is `compoundExpression` it parses any SPDX
license expression into a list of SPDX simple-expressions.
The rationale is Nixpkgs' license metadata isn't capable of
distinguishing between the AND and OR relationships.
License exceptions aren't currently taken into account.
* Add tests for the SPDX parser combinators
I simply added a file with expressions that are expected to fail to
parse or parse successfully.
* Add the SPDX license list
The SPDX license list as attrsets following the nixpkgs lib.licenses
convention.
This uses fetchurl which is not ideal because it's not available during
pure evaluation.
It does not yet include the SPDX exceptions list.
* Refactor cabal-licenses.nix to use spdx/licenses.nix
The handling of the generic licenses is undecided still. Some have been
removed because they have better official identifiers.
* Refactor license mapping in builders
The common code in the comp- and setup-builders has been extracted and
refactored to use the SPDX expression parser.
* Use spdx-license-list-data from nixpkgs
This conveniently solves the impurity problem with using fetchurl : )
I'm not sure threading `pkgs` through everything to get access to the
spdx license list package is the right way to go about this.
* hscolour to "LGPL-2.1-only" and remove "LGPL"
* Use evalPackages for spdx and move shim to overlay
* Better fix for LGPL packages.
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
* Adds support for cross package refs (with a project). Relative
directory references between packages within a project should now
work.
* Adds `includeSiblings` to `cleanSourceWith`. When `true` it
prevents the `subDir` arg from causing filtering of other directories.
* Adds `keepGitDir` to `cleanGit` to allow `.git` directory to be kept
(useful for components that use the `githash` package).
With cabal 3 it should be possible to reference a sublib in the
`build-depends` of a `.cabal` file using `package-name:sublib-name`
The `cabal-sublib` test is updated to include the new type syntax
and a fix is included for the component builder.
We will need to update `plan-to-nix` in `nix-tools` as well.
For now the work around is to use a module to add the sublib to
`depends` (see `test/cabal-sublib/default.nix`).
Without this fix the `cabal-sublib:slib` was not found by
`setup configure` because only `--dependency=slib=cabal-sublib-...`
was passed. The fix is to also pass
`--dependency=cabal-sublib:slib=cabal-sublib-...`.
After investigating adding a project argument to override the
mapping of libs to packages we realised we could fix the X11
package by updating `modules/configuration-nix.nix`.
We also updated `system-nixpkgs-map.nix` to allow multiple
packages to be mapped for a single library (turned out not
to be necessary for X11).
Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
Fixes ghcjs 8.6.5 and 8.8.4.
Uses emscripten for ghcjs 8.8 and adds scripts for building ghcjs 8.8.
Many fixes for ghcjs test failures.
Includes relocatableConfigFiles for making relocatable ghcjs bundles.
- Added the ability to generate coverage reports for packages and
projects.
- Outputs mix and tix information, as well as a HTML report.
- Added the "doCoverage" module option that allows users to choose
packages to enable coverage for.
- Added a "doCoverage" flag to the component builder that outputs HPC
information when coverage is enabled.
- Added the "overrideModules" library function to make it more
ergonomic fo users to enable coverage on existing projects.
- Modified the "check" builder to also output ".tix" files (if they
exist). This information is required to generate the coverage
report.
- Added a test for coverage.
* Changes to allow opencv to be built
The opencv library needs:
1. pkgconfig for opencv should return OpenCV 3, not OpenCV 2 (which is
what pkgs.opencv is)
2. hardeningDisable = [ "bindnow" ] (as per
https://github.com/LumiGuide/haskell-opencv/commit/5ea490af7662fe74b3740619963d37f1fd91e3d4#)
This commit fixes (1) by adding an entry to lib/pkgconf-nixpkgs-map.nix,
and partially fixes (2) by adding a new `hardeningDisable` option to the
set of package options. The user will still need to specify:
packages.opencv.hardeningDisable = [ "bindnow" ];
I'm not sure if there's a place to specify this as default in haskell.nix.
* Update comp-builder.nix
It causes a lot of issues. To make this work:
* `shellFor` is updated to use `getAllComponents` instead of `.all`.
* `getAllComponents` is updated to work on the package rather than
the package config.
* Tests updated to not use `.library` or `.exes.X` where appropriate.
* Documentation updated.
* Out of date examples removed.
As a bonus `shellFor` now takes a `components` argument that might be
useful for limiting the dependencies of the shell to just the ones
needed for the components you intend to work on.
Changes to the interface of haskell.nix (from the changelog.md file):
* Removed `sources.nixpkgs-default`, use `sources.nixpkgs` instead.
* Removed `./nixpkgs` directory, use `(import ./. {}).sources`
or `./nix/sources.nix` instead.
* Removes V1 interface for details on how to fix old code see:
https://github.com/input-output-hk/haskell.nix/issues/709
* Removed defaultCompilerNixName.
* cabalProject, cabalProject', hackage-project and hackage-package
now require a `compiler-nix-name` argument.
* `haskell-nix.tool` and `.tools` now require a `compiler-nix-name` argument.
New functions `p.tool` and `p.tools` (where p is a project) do not.
Like `shellFor { tools = ... }` they will use the compiler nix name
from the project (including stack projects where it is derived from
the resolver).
* `haskell-nix.alex` and `haskell-nix.happy` have been removed. Use
`p.tool "alex" "3.2.5"` or `shellFor { tools = { alex = "3.2.5"; } }`.
* `haskell-nix.nix-tools` -> `haskell-nix.nix-tools.ghc883` (it includes
the hpack exe now).
* `haskell-nix.cabal-install` ->
`p.tool "cabal" "3.2.0.0"` or `shellFor { tools = { cabal = "3.2.0.0"; } }`
* `haskell-nix.haskellNixRoots` -> `haskell-nix.roots ghc883` or `p.roots`
Other changes:
Adds hpack executable to the nix-tools derivations.
Adds a `cabal-hpack` test to make sure `hpack` works with
`cabalProject`.
Reduces the number of calls to `cabalProject` (particularly when
checking materialization), by giving internal tools a per-compiler
attribute.
Uses happy 1.19.12 when building newer ghc versions.
Updates cabal-install 3.2.0.0 to use the source from github that
is compatible with ghc 8.10.1.
Updates the docs for callCabalProjectToNix.
Adds a license mapping to fix a common warning.
Generation the Haddock documentation for a component in a dedicated
derivation (instead of as part of the derivation to build the component).
This means that the haddock documents will not be built until they are
required for instance in a shellFor shell.
Co-authored-by: Richard Wallace <richard.wallace@simspace.com>
This fixes and issue with TH functions like `makeRelativeToProject`
that can be used to keep a reference to source files. If they are in
a temporary location that can result in surprising `file not found`
errors at run time.
It may also be useful for making debug symbols and the like refer to
real files.
There is a risk that keeping the source for all packages could expand
closure sizes (if for instance debug info is wanted, but you do not
want to keep the source in the store). For that reason we have
made this an option that must be enabled.
* allow patchelf, stripping binaries, static building, dead code eliminiation, and other doc flags to be controlled via component options
* add ability to turn off shared libraries
This change replaces `passthru.run` and `collectRunComponents` with
`haskellLib.check` and `$pkg.checks`.
Uses `ln -s` instead of `cp` for DLLs to avoid duplicating them.
Adds features to help debugging eval issues with haskell.nix/release.nix.
The ifdInputLevel can be set to limit the jobs included to control
how much work is done at the eval stage when looking for issues.
Reduce CI times by excluding older ghc versions from haskellNixRoots
(GHC 8.6.3 and 8.6.4)
Sets `preferLocalBuild=false` on `callCabalProjectToNix` and `callStackToNix`
to ensure they are cached on hydra for both Linux and macOS.
Fix `data-dir:` when cleaning components.
`normalizeRelativeDir` adds a slash on to the end of `dataDir`.
Adding another one here results in `//` and files are left out by
mistake.
Changes to the haskell.nix code to fix broken tests:
* Add missing `then` to `call-cabal-project-to-nix.nix`.
* Fix default `hsSourceDirs` so that `.` gets included for `.all` component if one of the components does not have a `hsSourceDir` set.
* Fix `haskellNixRoots` so it works when cross compiling to windows.
Improvements to the haskell.nix tests:
* Run haskell.nix tests of nixpkgs 19.03 and 19.09.
* Run haskell.nix tests cross compiled to Windows under Wine (when possible).
* Add nix used as IFD inputs as tests to ensure they are cached.
* Use `haskell-nix.cabal-install` instead of `nixpkgs.cabal-install` in tests.
Without this fix `shellFor` fails for packages where a component has
a `preCheck` or `postCheck` option set. It should be safe to exclude
these because `shellFor` uses `phases = ["installPhase"]` and it seems
like a bad idea to use `all` for running tests (better to run the
individual `tests` components).
Currently the haskell.nix overlays replace `haskell` and
`haskellPackages` in the root of pkgs with versions that have just a
fraction of what nixpkgs provides.
Although `haskell-nix.haskellPackages` could be used to provide a much
more complete set there are subtle differences that are likely to cause
issues (and nixpkgs has patches for many haskell packages).
To make it possible for haskell.nix and nixpkgs haskell to coexist,
we need to move the remaining parts of haskell.nix outside of the
`haskell-nix` attribute into `haskell-nix`.
```
ghc -> haskell-nix.ghc
cabal-install -> haskell-nix.cabal-install
haskell.compiler -> haskell-nix.compiler
haskell.packages -> haskell-nix.packages
bootstrap.haskell.compiler -> haskell-nix.bootstrap.compiler
bootstrap.haskell.packages -> haskell-nix.bootstrap.packages
```
The following `haskellPackages` have been removed and instead the
nixpkgs versions will now be in `haskellPackages`. User code should
be updated if the nixpkgs versions are not desired:
```
haskellPackages.hoogle
-> haskell-nix.haskellPackages.hoogle.components.exes.hoogle
haskellPackages.happy
-> haskell-nix.haskellPackages.happy.components.exes.happy
haskellPackages.alex
-> haskell-nix.haskellPackages.alex.components.exes.alex;
```
Guard code has been added to overlays/defaults.nix to prevent us
inadvertently depending on the nixpkgs `haskell` or `haskellPackages`
inside the haskell.nix code base.
Projects like asterius and leksah that include dependencies on ghc. Sometimes the are needed by a setup.hs file (as is the case for the binaryen package used by asterius). The recent overlays merge broke this and #279 was needed to allow reinstallableLibGhc to be used to fix it. This PR adds a test case to make sure it is not inadvertently broken again. Also gives us a place to try out better ways to make this type of package work.* Test setup deps issue depending on ghc and Cabal
* Uses nixpkgs 19.09 + macOS wine fix on hydra/buildkite
* Enables macOS hydra tests
* disables musl tests for now
* Adds haskellNixRoots to tests
* Uses reinstallableLibGhc to fix the new setup-depends test
* Adds meta.platforms to haskell.nix components
* Set meta.platforms on cabal-install
* Builds some maintainer-scripts on hydra
* Pins hackage index-state for ghc-extra-packages
This reverts the reinstallable packages to before PR #261 was merged and
only flags the additional non-reinstallable packages as
non-reinstallable if reinstallableLibGhc is `false`.
The Overlays branch
This is a major reorganization in how haskell.nix is used, and marks our 1.0 release.
The branch doesn't build due to numerous issues that we believe to be with the CI
and not this branch. We expect only very minor adjustments prior to calling this the
official 1.0 release.
* Move iohk-nix patches into haskell.nix
This moves the customizations we did in iohk-nix
into haskell.nix via overlays and config.
Add bootPkgs logic
this moves the nuking of bootPkgs where it belongs. This should eventually still be
removed and replaced by a proper solution, that doesn't require the nuking of
bootPkgs.
Allow us to bootstrap binary ghcs and a cabal-install
With this we can do the following:
```
$ nix repl compiler/old-ghc-nix
nix-repl> :b (let pkgs = import <nixpkgs> {}; in with import ./. {}; import ./compiler/bootstrap/cabal-install.nix { inherit (pkgs) fetchurl stdenv zlib; inherit hackage
; ghc = ghc844; src = pkgs.fetchurl { url = "https://github.com/haskell/cabal/archive/Cabal-v3.0.0.0-rc3.tar.gz"; sha256 = "1zl2mgg8307ykq3v8nmafc6zdhhj1cw7w8ffpap16dsm6
5lbnx33"; }; })
```
which wile it may look daunting, will allow us to bootstrap a
cabal-install with a ghc. From that point onwards, we should be able to
build any hackage package via haskell.nix.
Pass through cabal-install version
Better threading of arguments.
Add bootstrap overlay
Allow alex + happy to be built
This still has a wart: we need nix-tools, and for that we use the ghc865 from
nixpkgs. Ideally we'd build nix-tools against a specific ghc, but then
we'd need a build expression for that.
Make ghcjs work
Building something like this:
```
nix build '(with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; })).components.exes.hello'
```
will now work. Assuming `nixpkgs` has been appropriately patched to support the `js-unknown-ghcjs` triple.
Also: this will need an additional `Cabal` patch, to make `Cabal` understand what it needs to do with: `dist/build/hello/hello: copyFile: does not exist (No such file or directory)`
It needs to learn that `dist/build/hello/hello.jsexe` is what it wants to copy and that that is a directory. Luckily we do have some code in Cabal already that does this for `.exe` on windows.
Build `js-unknown-ghcjs` packages with haskell.nix
Using the following expression:
```
with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; };
let Cabal = buildPackages.haskell-nix.hackage-package {
name = "Cabal"; version = "2.4.1.0";
modules = [
{ packages.Cabal.patches = [ ./Cabal-install-folder.diff ]; }
];
}; in
(haskell-nix.hackage-package {
name = "hello"; version = "1.0.0.2";
modules = [
({config, ... }:{ packages.hello.package.setup-depends = [ Cabal ]; })
];}).components.exes.hello
```
in a `test.nix` file. And running
```
nix build -f ./test.nix
```
on it, will produce
```
./result
├── bin
│ └── hello.jsexe
│ ├── all.js
│ ├── all.js.externs
│ ├── index.html
│ ├── lib.js
│ ├── manifest.webapp
│ ├── out.frefs.js
│ ├── out.frefs.json
│ ├── out.js
│ ├── out.stats
│ ├── rts.js
│ └── runmain.js
└── share
└── doc
└── x86_64-linux-ghc-8.6.5
└── hello-1.0.0.2
└── LICENSE
6 directories, 12 files
```
our change from string -> str to evade the warnings nix generates, resulted in a bunch of
unforseen consequences.
We now modify the str type a bit for our own use.
Uses the `isLocal` for property set by `plan-to-nix` and `stack-to-nix`
on packages where the source is referenced in the `cabal.project`
or `stack.yaml` file.
* Filter components using buildable flag
* Bump nix-tools
* Include components without buildable flag
* Test buildable components are filtered correctly
Hamish Mackenzie
Ollie CharlesandMichael Peyton Jones
Michael Peyton Jones
toonnandHamish Mackenzie
Samuel Evans-Powell
John A. Lotoski
Silvan Mosberger
Jean-Baptiste GiraudeauandMoritz Angermann
Yorick
Viktor Kronvall
Matthew Ess