* Use nix module system to check project arguments
This change uses the nix module system for the arguments to:
* `cabalProject'`
* `cabalProject`
* `stackProject'`
* `stackProject`
This will:
* Give an error when the name or type of an argument is incorrect
(currently misnamed args are quietly ignored).
* Fixes a problem with `projectCross` not having a way to pass different args to `cabalProject`
for different platforms. We can now use:
`({pkgs, ...}: { cabalProjectLocal = if pkgs.stdenv.hostPlatform.isX then ... })`
* Add test.
* wip
* Fix merge regression
* Combine drv, drv.source and drv.dist for doctest
* Skip cabal-doctests test when cross compiling
* Add the --no-magic flag to the .cabal file for the cabal-doctests test.
This appears to be necessary on OSX.
The --no-magic flag stops the doctest executable from expanding path
arguments, trying to locate the package db, etc.
This shouldn't be necessary with cabal-doctest, since all the
necessary options to pass to doctest are computed when running the Setup.hs
script.
See
https://github.com/input-output-hk/haskell.nix/pull/427#issuecomment-605761780.
* Fix cabal-doctest support
* Skip cabal-doctest test plan cross compiling
* More fixes for cabal-doctest
* Skip cabal-doctest tests when cross compiling
Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
Co-authored-by: Hamish Mackenzie <Hamish.K.Mackenzie@gmail.com>
Currently in order to support submodules all the directories are included (but not all files) when building the whitelist. This is done because we need the `.git` files (or directories) the contain to be present when we run `git ls-files --recurse-submodules`.
Including every directory causes two performance issues:
* Creating a directory in your repo (even empty one not added to git at all) will require the whitelist to be recalculated.
* Recalculation of the whiltelist may be slowed down if there are a large number of directories. In particular `lstat` is called on every file (probably to get the right `type` to pass to the filter function).
The fix is to use `git submodule status --recursive` to get a list of all the submodule directories. Then the `.git` files (or directories) in them can be included. This has to be repeated until all the submodules are known (as `--recursive` will not see the submodules until the parent module is included).
It looks like `callCabalProjectToNix` can fail in its `cabal` call with the following error:
```
open: permission denied (Permission denied)
```
Then running the same derivation locally works. If the error is deferred there is no easy way on hydra to retry.
This change makes it so only solvers errors (which should hopefully be deterministic) are deferred. All other errors will go back to being eval time errors meaning they should be automatically retried at the next hydra eval.
* Use emscripten branch of GHCJS for 8.6
* Adds release-linux-only.nix (for hydra with no mac builders)
* Sets HOME to a temp dir once per derivation to help performance of emcc
* Updates materialization
* Uses symlinks to reduce ghc derivation size for wrapped ghcjs
* Removes compiler-nix-name test
This test checked that the default compiler was overridden by the
compiler-nix-name arg. There is no default any more.
The GHCJS branches used include fixes for:
* hs$ret_1 typo (should be hs$ret1)
* A bignum issue
* Delays cabal configure errors until after evaluation when possible.
* Avoids rerunning long `cabal configure` if we already know it will fail.
* Provides a way to get a component from a package or project using a `cabal` like reference.
Code using the `tool` functions will automatically use `getComponent`.
For `(hackage-package {...}).components.library` is also ok.
```
# Consider changing hackage-package use:
(pkgs.haskell-nix.hackage-package {...}).components.exes.something
(pkgs.haskell-nix.hackage-package {...}).getComponent "exe:something"
# For any cabal project:
project.hsPkgs.somepackage.components.exes.something
project.getComponent "somepackage:exe:something"
# or do it in two steps
(project.getPackage "somepackage").getComponent "exe:something"
```
The reason for the new function is that we cannot provide the attribute interface without knowing that packages are in the project first.
Here is how the `cabal configure` error output is handled:
* The `plan-nix` derivation builds even if `cabal configure` fails.
* When it fails, it copies `failed-cabal-configure.nix` to the `$out/default.nix` along with a copy of the `cabal configure` output.
* When `failed-cabal-configure.nix` is imported and used in any way it writes the `cabal configure` output with `__trace` so it will always be visible.
* Instead of a `plan` the imported nix contains a `configurationError` pointing the `cabal configure` output.
* The intermediate functions `configurationError` and bubble it up to where it is needed.
* `getPackage` returns a mostly empty proxy for a real package when there is a `configurationError`
* The `getComponent` function always returns a derivation, but the version in the proxy writes the `cabal configure` output to stdout and calls `exit 1` (so that it will never build).
Issues with `cabal.project.freeze` files can be tricky to debug.
While it is possible to run without a freeze file and look
at the `plan-nix` or `plan-nix.json` (the `plan.json`) created when
haskell.nix configured the project, it would be nice to
also have the output of `cabal freeze` as it will be in the
correct format and contains a different subset of `plan.json`
than the `plan-nix`.
This might also be useful for automating updating of `.freeze`
files.
This combined with setting the `use-pkg-config` flag on `postgresql-libpq` should get it compiling. There is still a problem with `openssl` that requires a `module` to fix the linking of executables.
Add the following to the `cabal.project` file or to `cabalProjectLocal`:
```
package postgresql-libpq
flags: +use-pkg-config
```
Include a module like this one to include `openssl` in the linker arguments:
```nix
{
modules = [(
{pkgs, ...}: final.lib.mkIf pkgs.stdenv.hostPlatform.isMusl {
# The order of -lssl and -lcrypto is important here
packages.postgrest.configureFlags = [
"--ghc-option=-optl=-lssl"
"--ghc-option=-optl=-lcrypto"
"--ghc-option=-optl=-L${pkgs.openssl.out}/lib"
];
})];
}
```
This PR also adds a "hackage quirk" for `postgrest` to that makes these changes for `postgrest` automatically when built it as a `tool` or with `hackage-package`.
Unfortunately we do not have a good way to avoid the need make these changes.
When a package is not in a subdirectory, but instead
in the root of the project the cleaning of the package
source was broken.
This might explain why some `cabal.project` file
based projects might also see be affected by #1013.
See https://github.com/input-output-hk/nix-tools/pull/101
We should look up hsPkgs.${pkg-name}.components.exe.${component-name}
instead of including the package as a dependency.
Adding `hsPkgs.${pkg-name}` a tool it is not clear which executable in the package `haskell.nix` should choose.
Haskell.nix did not deal with this well and in fact it could lead to `nix-shell` crashing. For instance using `shellFor` to make a shell for building `haskell-language-server` with `cabal build` crashed as a dependency on `ghcide` the executable (by the `ghcide-bench` executable) caused infinite recursion. Allowing `ghcide-bench` to correctly depend just on `components.exes.ghcide` fixes this.
This PR also includes:
* Updated materialized files built with the new nix-tools and latest index-state
* Small update to GHCJS (pins happy) to fix issue that showed up when the materialization was updated
* A fix for the infinite recursion issue when updating materialized files (by adding ghc-boot-packages-unchecked).
* Performance fix for shellFor (checks just the names of component derivations when filtering).
The builtins.fetchGit used by haskell.nix clones only the subset of a
repository corresponding to HEAD. So if we have a workflow that has a
branch "development" as HEAD, but the stack.yaml only targets commits
from "master", then it will be unable to fetch any commit.
Fixes#592
The [dependent-map](https://hackage.haskell.org/package/dependent-map)
package for instance is not clear on which free license applies to
which parts of the code, but that does not mean it is not free.
`cabal2nix` maps LicenseRef-OtherLicense to `meta.license = "unknown"`
`cabal-to-nix` and `haskell.nix` map it to a more detailed attribute
set but it includes `free = false` and that causes packages like
`dependent-map` to fail to install when `allowUnfree` has not been set.
This change removes the `free` attribute altogether, which seems more
principled than asserting that the `OtherLicense` cannot be `free`.
I think this is more correct: other package that depend on the system
odbc package may not need freetds, the problem is that the Haskell odbc
package doesn't declare its dependency on freetds.
* Adding odbc alias to the unixODBC package
* Adding odbc alias to the unixODBC package
* Adding odbc alias to the unixODBC package
* Adding comments about odbc package dependencies
* Add parser combinators to parse SPDX license expressions
The most important parser is `compoundExpression` it parses any SPDX
license expression into a list of SPDX simple-expressions.
The rationale is Nixpkgs' license metadata isn't capable of
distinguishing between the AND and OR relationships.
License exceptions aren't currently taken into account.
* Add tests for the SPDX parser combinators
I simply added a file with expressions that are expected to fail to
parse or parse successfully.
* Add the SPDX license list
The SPDX license list as attrsets following the nixpkgs lib.licenses
convention.
This uses fetchurl which is not ideal because it's not available during
pure evaluation.
It does not yet include the SPDX exceptions list.
* Refactor cabal-licenses.nix to use spdx/licenses.nix
The handling of the generic licenses is undecided still. Some have been
removed because they have better official identifiers.
* Refactor license mapping in builders
The common code in the comp- and setup-builders has been extracted and
refactored to use the SPDX expression parser.
* Use spdx-license-list-data from nixpkgs
This conveniently solves the impurity problem with using fetchurl : )
I'm not sure threading `pkgs` through everything to get access to the
spdx license list package is the right way to go about this.
* hscolour to "LGPL-2.1-only" and remove "LGPL"
* Use evalPackages for spdx and move shim to overlay
* Better fix for LGPL packages.
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
* Adds support for cross package refs (with a project). Relative
directory references between packages within a project should now
work.
* Adds `includeSiblings` to `cleanSourceWith`. When `true` it
prevents the `subDir` arg from causing filtering of other directories.
* Adds `keepGitDir` to `cleanGit` to allow `.git` directory to be kept
(useful for components that use the `githash` package).
After investigating adding a project argument to override the
mapping of libs to packages we realised we could fix the X11
package by updating `modules/configuration-nix.nix`.
We also updated `system-nixpkgs-map.nix` to allow multiple
packages to be mapped for a single library (turned out not
to be necessary for X11).
Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
I'm wondering how/if this worked before, i couldn't find most xorg
packages on their previous path.
(Neither on the current version of nixpkgs nor on nixpkgs anno 2018)
* Coverage: allow projects with no libraries
- Prevent projects without a library from breaking the coverage report
generators.
- Added a test to ensure projects without libraries can build coverage
reports.
* Ensure the correct GHC version is used in project coverage reports
- Previously, when trying to determine the version of GHC to use for a
project coverage report, we would attempt to find the version of GHC
used in the constituent coverage reports, or default to the GHC that
is shipped with Nixpkgs. Using the GHC that ships with Nixpkgs is
problematic (specifically the coverage-no-libs tests fail to run on
Hydra), so we now ask the user to pass in the project to the
"projectCoverageReport" function so we can find the correct version of
GHC to use in a foolproof manner.
* Fix stack.yaml filtering when resolver file is in a subdirectory
`maybeCleanedSource` tries to include the resolver file into the cleaned
source, if a resolver is specified, but it was only working when the
file was in the root of the project, because all directories would be
filtered out.
* fixup! Fix stack.yaml filtering when resolver file is in a subdirectory
Fix calculating relPath for paths using subDir
* Add test case for local stack snapshot located in a subdirectory
* Remove unconditional trace for index-state
We already have messages for unusual cases (like using the latest
index-state). This message is unconditional, so quite noisy.
We include the actual index-state in the builer log so it's available if you
want to check it.
* Move source-cleaning warning to builder log
It's not *that* interesting, and someone who cares will likely check the
log.
* Move shellFor trace message to builder log
Not that interesting, really.
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
* Drop 19.09 support
This is a year old. NixOS doesn't even support the last stable release,
let alone the one before that.
Also cuts our CI size by a third, which is always nice.
* Remove 19.09 compatibility pieces
Fixes ghcjs 8.6.5 and 8.8.4.
Uses emscripten for ghcjs 8.8 and adds scripts for building ghcjs 8.8.
Many fixes for ghcjs test failures.
Includes relocatableConfigFiles for making relocatable ghcjs bundles.
* Do not use evalPackages to create sourceRepo for builds
but still use it when creating source repo for plan calculation.
This might be a way to fix#917
* Update unit tests
* Refactor this patch, and add some comments
this is much less convoluted. I moved the fetchgit invocation from
`cabal-project-parser.nix` to `call-cabal-project-to-nix` (which makes
sense in a way, if a parser should return parsed data, and not do
stuff), and then some simplifications could happen.
Also added some comments.
* Update unit tests
* Dead code
* Less repetition in tests
* Comment tweaks
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
- Added the ability to generate coverage reports for packages and
projects.
- Outputs mix and tix information, as well as a HTML report.
- Added the "doCoverage" module option that allows users to choose
packages to enable coverage for.
- Added a "doCoverage" flag to the component builder that outputs HPC
information when coverage is enabled.
- Added the "overrideModules" library function to make it more
ergonomic fo users to enable coverage on existing projects.
- Modified the "check" builder to also output ".tix" files (if they
exist). This information is required to generate the coverage
report.
- Added a test for coverage.
* Changes to allow opencv to be built
The opencv library needs:
1. pkgconfig for opencv should return OpenCV 3, not OpenCV 2 (which is
what pkgs.opencv is)
2. hardeningDisable = [ "bindnow" ] (as per
https://github.com/LumiGuide/haskell-opencv/commit/5ea490af7662fe74b3740619963d37f1fd91e3d4#)
This commit fixes (1) by adding an entry to lib/pkgconf-nixpkgs-map.nix,
and partially fixes (2) by adding a new `hardeningDisable` option to the
set of package options. The user will still need to specify:
packages.opencv.hardeningDisable = [ "bindnow" ];
I'm not sure if there's a place to specify this as default in haskell.nix.
* Update comp-builder.nix
Hamish Mackenzie
ismaelbouyaf
Alexander Vieth
Jakub Kopański
Michael Peyton Jones
Alexander Shestakov
toonnandHamish Mackenzie
paumr
Samuel Evans-Powell
Zhenya Vinogradov
Thomas Tuegel
Marc Fontaine
Joachim BreitnerandHamish Mackenzie
John Ky
Ollie Charles
John A. Lotoski