This adds a way to specify the `evalSystem` or `evalPackages` explicitly when calling the `project` functions.
Currently if we want to make a `flake` that supports multiple systems we have few options:
* Require builders for all the supported systems (even just for `nix flake show`).
* Pass `--impure` so that haskell.nix can see `builtins.currentSystem` to set up `pkgs.evalPackages` to use that. Unfortunately this prevents nix from caching some of the work it does and often results in it recalculating for each supported system when it would otherwise be cached and take no time at all.
* Add an overlay to replace `evalPackages`. This works, but it is not straight forward.
* Materialize the nix files for the project.
This change allows `evalSystem = "x86_64-linux";` to be passed telling `haskell.nix` to run `cabal` and `nix-tools` on that system. The user will have to have a builder for that system, but does not need to have builders for the others (unless building outputs for them).
Fixes Windows cross compilation for GHC 9.0 and 9.2.
Updates wine to use the version that is included with the chosen Nixpkgs (it used to be pinned to an older version).
Moves the configuration of the GHC source into the GHC derivation. This should make adding Hadrian support easier.
Propagates library dependences (not just pkgconfig ones) on windows so that any DLLs in those libraries can be copied for TH evaluation and to the `/bin` directory of executable components.
Adds gcc and mfcgthreads as library dependencies on Windows so that the DLLs they include will be found.
Use `$pkgsHostTarget` (instead of `ghc-pkg`) to find all the DLLs can copy them to the `/bin` directory of executable components.
Adds support for __int128_t and __uint128_t to language-c to fix aarch64-darwin builds.
Fixed reinstalling packages that come with patched versions in ghcjs.
This is a better fix to the problem of making DLLs available to the process running in wine that is used to evaluate TH code when cross compiling for Windows.
Also fixes macOS loadArchive in TH for ghc 9
Co-authored-by: Michael Peyton Jones <michael.peyton-jones@iohk.io>
* Improve support for external Hackage repositories
This change builds #535. `repository` blocks in `cabal.project` parsed and `cabal` is used to automatically downloaded them. Then `hackage-to-nix` is used to produce the nix required.
To make it work with restricted eval (on hydra for instance) we need to include a sha256 like this:
```
repository ghcjs-overlay
url: https://input-output-hk.github.io/hackage-overlay-ghcjs
secure: True
root-keys:
key-threshold: 0
--sha256: sha256-EPlLYPmIGtxeahlOspRzwJv+60N5mqrNC2BY4jZKceE=
```
To find the correct `sha256` put in an invalid one and attempt a build.
* Include dependencies of haskell.nix that were tracked in `nix/sources.json`
as flake inputs (`flake.lock` replaces `nix/sources.json`).
* Use `flake-compat` to continue to provide a compatible interface for non
flake projects.
Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
Co-authored-by: Shea Levy <shea@shealevy.com>
* Add test.
* wip
* Fix merge regression
* Combine drv, drv.source and drv.dist for doctest
* Skip cabal-doctests test when cross compiling
* Add the --no-magic flag to the .cabal file for the cabal-doctests test.
This appears to be necessary on OSX.
The --no-magic flag stops the doctest executable from expanding path
arguments, trying to locate the package db, etc.
This shouldn't be necessary with cabal-doctest, since all the
necessary options to pass to doctest are computed when running the Setup.hs
script.
See
https://github.com/input-output-hk/haskell.nix/pull/427#issuecomment-605761780.
* Fix cabal-doctest support
* Skip cabal-doctest test plan cross compiling
* More fixes for cabal-doctest
* Skip cabal-doctest tests when cross compiling
Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
Co-authored-by: Hamish Mackenzie <Hamish.K.Mackenzie@gmail.com>
* Use emscripten branch of GHCJS for 8.6
* Adds release-linux-only.nix (for hydra with no mac builders)
* Sets HOME to a temp dir once per derivation to help performance of emcc
* Updates materialization
* Uses symlinks to reduce ghc derivation size for wrapped ghcjs
* Removes compiler-nix-name test
This test checked that the default compiler was overridden by the
compiler-nix-name arg. There is no default any more.
The GHCJS branches used include fixes for:
* hs$ret_1 typo (should be hs$ret1)
* A bignum issue
* Add `.debug` to build any component with DWARF dugug info on linux
(ghc >=8.10.2).
* Pass `enableDWARF` to `shellFor` for to get a shell where all the
components are the `.debug` ones.
* Adds support for cross package refs (with a project). Relative
directory references between packages within a project should now
work.
* Adds `includeSiblings` to `cleanSourceWith`. When `true` it
prevents the `subDir` arg from causing filtering of other directories.
* Adds `keepGitDir` to `cleanGit` to allow `.git` directory to be kept
(useful for components that use the `githash` package).
* Add golden coverage test
- Other coverage tests check that the coverage tests can be generated,
but don't check the coverage output itself. This commit adds a test
to ensure that the coverage output is correct.
* Use niv and haskell.nix for hpc-coveralls
* Get `sources` from `haskell-nix.callPackages`
* Use `builtin: false` for hydra in sources.json
* Fix for cross compilation.
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
* Coverage: allow projects with no libraries
- Prevent projects without a library from breaking the coverage report
generators.
- Added a test to ensure projects without libraries can build coverage
reports.
* Ensure the correct GHC version is used in project coverage reports
- Previously, when trying to determine the version of GHC to use for a
project coverage report, we would attempt to find the version of GHC
used in the constituent coverage reports, or default to the GHC that
is shipped with Nixpkgs. Using the GHC that ships with Nixpkgs is
problematic (specifically the coverage-no-libs tests fail to run on
Hydra), so we now ask the user to pass in the project to the
"projectCoverageReport" function so we can find the correct version of
GHC to use in a foolproof manner.
* Fix stack.yaml filtering when resolver file is in a subdirectory
`maybeCleanedSource` tries to include the resolver file into the cleaned
source, if a resolver is specified, but it was only working when the
file was in the root of the project, because all directories would be
filtered out.
* fixup! Fix stack.yaml filtering when resolver file is in a subdirectory
Fix calculating relPath for paths using subDir
* Add test case for local stack snapshot located in a subdirectory
Fixes ghcjs 8.6.5 and 8.8.4.
Uses emscripten for ghcjs 8.8 and adds scripts for building ghcjs 8.8.
Many fixes for ghcjs test failures.
Includes relocatableConfigFiles for making relocatable ghcjs bundles.
This change introduces a `ghc8102-experimental` compiler-nix-name that behaves much like the `ghc8102` version but includes @JoshMeredith's patch `./patches/ghc/ghc-8.10-ubxt.patch` that writes the `cg_binds` field with:
```haskell
liftIO $ registerInterfaceDataWith "ghc/phase/core" hsc_env $ \bh ->
putWithUserData (const $ return ()) bh (map toIfaceBind $ cg_binds cg_guts)
```
Some of the places where we used to depend on the output of `plan-to-nix` containing the correct `nix-name` for the compiler have been update to avoid using incompatible interface files from the `compiler-nix-name = "ghc8102"` version of GHC.
- Added the ability to generate coverage reports for packages and
projects.
- Outputs mix and tix information, as well as a HTML report.
- Added the "doCoverage" module option that allows users to choose
packages to enable coverage for.
- Added a "doCoverage" flag to the component builder that outputs HPC
information when coverage is enabled.
- Added the "overrideModules" library function to make it more
ergonomic fo users to enable coverage on existing projects.
- Modified the "check" builder to also output ".tix" files (if they
exist). This information is required to generate the coverage
report.
- Added a test for coverage.
When ghc 8.8.4 and ghc 8.10.2 were added we stopped building the older
versions on ci. This changes adds them back but does not run all
the tests on them.
Changes to the interface of haskell.nix (from the changelog.md file):
* Removed `sources.nixpkgs-default`, use `sources.nixpkgs` instead.
* Removed `./nixpkgs` directory, use `(import ./. {}).sources`
or `./nix/sources.nix` instead.
* Removes V1 interface for details on how to fix old code see:
https://github.com/input-output-hk/haskell.nix/issues/709
* Removed defaultCompilerNixName.
* cabalProject, cabalProject', hackage-project and hackage-package
now require a `compiler-nix-name` argument.
* `haskell-nix.tool` and `.tools` now require a `compiler-nix-name` argument.
New functions `p.tool` and `p.tools` (where p is a project) do not.
Like `shellFor { tools = ... }` they will use the compiler nix name
from the project (including stack projects where it is derived from
the resolver).
* `haskell-nix.alex` and `haskell-nix.happy` have been removed. Use
`p.tool "alex" "3.2.5"` or `shellFor { tools = { alex = "3.2.5"; } }`.
* `haskell-nix.nix-tools` -> `haskell-nix.nix-tools.ghc883` (it includes
the hpack exe now).
* `haskell-nix.cabal-install` ->
`p.tool "cabal" "3.2.0.0"` or `shellFor { tools = { cabal = "3.2.0.0"; } }`
* `haskell-nix.haskellNixRoots` -> `haskell-nix.roots ghc883` or `p.roots`
Other changes:
Adds hpack executable to the nix-tools derivations.
Adds a `cabal-hpack` test to make sure `hpack` works with
`cabalProject`.
Reduces the number of calls to `cabalProject` (particularly when
checking materialization), by giving internal tools a per-compiler
attribute.
Uses happy 1.19.12 when building newer ghc versions.
Updates cabal-install 3.2.0.0 to use the source from github that
is compatible with ghc 8.10.1.
Updates the docs for callCabalProjectToNix.
Adds a license mapping to fix a common warning.
This change allows a resolverSha256 to be passed to stackProject.
When provided the resolver found in the stack.yaml is fetched
using this sha256 and the uri string is replaced with the
path to the fetched version.
Generation the Haddock documentation for a component in a dedicated
derivation (instead of as part of the derivation to build the component).
This means that the haddock documents will not be built until they are
required for instance in a shellFor shell.
Co-authored-by: Richard Wallace <richard.wallace@simspace.com>
This is a simple way to specify the sha256 values for references
to git repositories that do not have a `--sha256:` comment in the
`cabal.project` file or a `# nix-sha256:` comment in the `stack.yaml`
file.
The tests demonstrate how to use `sha256map` with both
`cabalProject` and `stackProject` to build the
haskell-language-server from github with the sha256 hashes
needed to work with `--option restrict-eval true`.
Fixes#672 by using the first index-state that is > than desired
index state. It also passes `--index-state` to `cabal v2-cconfigre`
to make sure it cabal uses only the packages in the hackage
file that are before the specified date.
Fixes#540 by returning the `index-state` used back out of
`cabalProject` in the same way `plan-nix` is returned.
* ghc 8.10
* Adds/Updates/Removes materialization files
* Disable one-shot-kqueue-on-macos patch on ghc-8.10
* Test ghc883 and ghc8101 but only with nix 20.03
* Various smaller fixes.
This should fix the infinite recursion issue with #654
Co-authored-by: Lennart Spitzner <hexagoxel@hexagoxel.de>
* Add compiler-nix-name to cabalProject
Setting the compiler explicitly does not make it easy for cross
compilation to also choose the correct compiler when compiling
build tools for the build platform. Also it is easy to pass the
wrong compiler type. Specifying just the nix-name of the compiler
makes that less likely.
Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
* Add support for external Hackage repositories
Currently haskell.nix is not able to build Cabal projects that depend on
packages from private Hackage repositories, as it make only main Hackage
available to cabal. This is unfortunate.
This commit adds this functionality, by allowing the user to pass
`extra-hackages` and `extra-hackage-tarballs` to `mkPkgSet` and
`callCabalToNix` respectively, to add as much extra repositories as
needed.
This repositories are first made available to Cabal when calling
`v2-configure`, resulting in correct plans. Later they are combined with
global Hackage when building dependencies of the local packages.
* Use cabal.project.freeze if available
Currently callCabalProjectToNix does not copy `cabal.project.freeze`
from source directory, leading to different build plans when building
components with nix and when building project with `cabal new-build`
inside `nix-shell`.
This behavior is undesired, so this commits fixes it.
* Add tests for extra-hackages functionality
Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
* Downloads using `builtins.fetchGit` when no sha is found.
* Adds a `lookupSha256` arg for when we can't change `cabal.project`.
* Outputs a message when the sha256 is missing that includes advice on how add one.
* Include `tests` in path for cross compiled tests
* Add ghc-boot-packages-nix to haskellNixRoots
* Fix naming warnings in tests sources
* Uses a single cleanGit call for all the tests
* Add memoize code for ghc-boot and ghc-extra pkgs
* Memoize ghc-boot and ghc-extra pkgs nix
* Fix release.nix being more strict than ci.nix by updating filterAttrsOnlyRecursive in ci-lib.nix
* Nicer errors when materialized path does not exist
* Updated materialization docs
* Add internalHackageIndexState to set the index-state used within haskell.nix
* Remove unused (and broken) test files
* Make default.nix expose pinned nixpkgs
Also make `nixpkgs/default.nix` just an attribute set of sources.
This is simpler, and consistent with what e.g. `niv` does.
* Add backwards compatibility shim, and version argument to allow evolution in future
* Fix some missed things
* Fix and improve quickstart
If the .nix files listed in the cache are full paths pointing at the
store we get errors like this in restricted eval mode:
error: access to path '/nix/store/...-cabal-simple.nix' is forbidden
in restricted mode
This change replaces `passthru.run` and `collectRunComponents` with
`haskellLib.check` and `$pkg.checks`.
Uses `ln -s` instead of `cp` for DLLs to avoid duplicating them.
Adds features to help debugging eval issues with haskell.nix/release.nix.
The ifdInputLevel can be set to limit the jobs included to control
how much work is done at the eval stage when looking for issues.
Reduce CI times by excluding older ghc versions from haskellNixRoots
(GHC 8.6.3 and 8.6.4)
Sets `preferLocalBuild=false` on `callCabalProjectToNix` and `callStackToNix`
to ensure they are cached on hydra for both Linux and macOS.
Fix `data-dir:` when cleaning components.
`normalizeRelativeDir` adds a slash on to the end of `dataDir`.
Adding another one here results in `//` and files are left out by
mistake.
Changes to the haskell.nix code to fix broken tests:
* Add missing `then` to `call-cabal-project-to-nix.nix`.
* Fix default `hsSourceDirs` so that `.` gets included for `.all` component if one of the components does not have a `hsSourceDir` set.
* Fix `haskellNixRoots` so it works when cross compiling to windows.
Improvements to the haskell.nix tests:
* Run haskell.nix tests of nixpkgs 19.03 and 19.09.
* Run haskell.nix tests cross compiled to Windows under Wine (when possible).
* Add nix used as IFD inputs as tests to ensure they are cached.
* Use `haskell-nix.cabal-install` instead of `nixpkgs.cabal-install` in tests.
Yvan Sraka
Hamish Mackenzie
Andrea Ciceri
Shea Levy
Moritz Angermann
Hamish Mackenzie
Dennis Gosnell
Samuel Evans-Powell
Zhenya Vinogradov
Maxim Koltsov
Michael Peyton Jones
Kirill Elagin