71 Commits

Author SHA1 Message Date
Hamish Mackenzie 9a0629494e Add ghc 9.4.4 materialisation and trace hydra job names (#1849)
* Trace the attribute names of hydra jobs

* Materialize hadrian plan-nix

* Only break out linux ghc 8.10.7 tests to cut down on the number of hydra jobs.

* Materialize iserv-proxy plan-nix
2023-02-19 20:43:52 +13:00
Andrea Bedini 7075077d46 Support revisions and more (#1775)
* Support revisions

Cabal files are now obtained from the index tarball (as cabal does)
directly. This allows us to always pick the cabal file revision that
cabal would pick, without having to understand or reference hackage.

* Update nix-tools materialised plan for new tooling

* Update materialized files

* Eval on x86_64-linux

* Turn off full materialization check

* Replace pkgs.writeText with passAsFile

Nix derivations are already able to pass a attribute to the builder as a
file. This means we don't need to use pkgs.writeText to turn the
cabalFile attribute into a file, saving one derivation.

* Bump closure size limit to 540

---------

Co-authored-by: Hamish Mackenzie <Hamish.K.Mackenzie@gmail.com>
2023-02-09 12:55:40 +13:00
Hamish Mackenzie 51025ec0c3 Try running tests on self-hosted runners (#1812)
* Try running tests on self-hosted runners

* Update pipeline.yml

* Update pipeline.yml

* Update pipeline.yml

* Update pipeline.yml

* Update check.nix

* Does this fix check-hydra?

* We need a ghc to unpack... I guess.

Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
2022-12-12 12:30:56 +13:00
Hamish Mackenzie cbf1e918b6 Add ghc 9.2.5 (#1779)
* Add ghc 9.2.5

* More materialization updates

* More materialization updates

* More materialization updates

* Remaining materialization changes

* Fix ghcjs cabal materialization issue
2022-11-15 17:12:14 +13:00
Yvan Sraka 8309423b1a Fix #518: examples in documentation need to be tested (#1714) 2022-10-06 10:36:38 +02:00
Shea Levy ed55264287 Update scripts: Use shallow clones for faster bootstrapping (#1733) 2022-10-03 14:39:21 +01:00
Hamish Mackenzie 64e8460c35 Bump nixpkgs pins (#1710)
* Fix ghcjs for time_t is 64bit in emscripten now.
* Fix error: qca2 has been removed, because it depended on qt4.
* Add spdx 3.18 materialization.
* Turn off building tests for wine-5.4 (due to IDL issue).
* Fix options-doc.nix.
* Comment added to force rebuild on hydra
2022-09-27 13:00:15 +13:00
Hamish Mackenzie 4c8bbbd3ed Add flake.variants (#1661)
* Add flake.variants

This allows flakes to easily include variations of the
project by with different project arguments.
Anything you can pass to `project.addModule` can be used.
For instance to include variants using ghc 9.2.4:

```
flake.variants.ghc924.compiler-nix-name = pkgs.lib.mkForce "ghc924";
```
Then use it with:

```
nix build .#ghc924:hello:exe:hello
```

This change also:

* Exposes add `combineFlakes` and `rawFlake` for more control over how flakes are combined.

* Includes package coverageReports to the `hydraJobs` of the flakes.

* Tries to make the materialisation concurrency test more reliable
2022-09-16 22:21:54 +12:00
Hamish Mackenzie a197fd0afb Add cabal-issue-8352-workaround (#1671)
* Add cabal-issue-8352-workaround

See https://github.com/haskell/cabal/issues/8352

* More script fixes

* More script fixes
2022-09-15 17:13:05 +12:00
Hamish Mackenzie 6c7f9d0042 Use cabal 3.8 (#1641)
A bug was fixed in cabal 3.8 (https://github.com/haskell/cabal/issues/6771) that haskell.nix relied on for `pkg-config` in support.  To work around this we added a dummy `pkg-config` that `cabal configure` now uses when haskell.nix runs it `cabal configure` internally.  That returns version information for all the `pkg-config` packages in `lib/pkgconfig-nixpkgs-map.nix`.  This mapping has also been expanded based on the results of searching nixpkgs for `*.pc` files.

This update also includes workarounds for:

https://github.com/haskell/cabal/issues/8352
https://github.com/haskell/cabal/issues/8370
https://github.com/haskell/cabal/issues/8455
2022-09-13 12:00:04 +12:00
Hamish Mackenzie c83b33f419 Split checking hydra eval on buildkite (#1633) 2022-08-31 12:15:24 +02:00
Hamish Mackenzie 53cf60c83c Use nixpkgs 22.05 for scripts (#1632)
This should daily updates which are failing with:

```
error: unrecognised flag '--accept-flake-config'
```

See https://buildkite.com/input-output-hk/haskell-dot-nix-nightly-updates/builds/1318#0182f20e-bd2b-4840-a930-af2815df793c

This change also updates to the newer mkdocs in nixpkgs 22.05 and updates the check-path-support to use a combination of nixpkgs and ghc that should be in the hydra cache.
2022-08-31 17:47:50 +12:00
Hamish Mackenzie e1b76b23b8 Fix typo (#1631) 2022-08-31 15:59:35 +12:00
Yvan Sraka a2a29bd368 Use nix --accept-flake-config option rather than yes | in CI tests (#1626)
https://nixos.org/manual/nix/unstable/command-ref/conf-file.html#conf-accept-flake-config
2022-08-30 11:30:09 +12:00
Hamish Mackenzie e07b0fadd2 Fix daily updates of hackage and stackage (#1622)
These were broken by #1579 because `nix flake lock` now asks for confirmation that it is ok to use the nix configuration in the flake.
2022-08-29 12:09:56 +12:00
Hamish Mackenzie c2f14344f1 Add ghc 9.2.4 (#1566) 2022-07-31 21:41:50 +12:00
Hamish Mackenzie 2739817902 Fixes for materialization (#1514) 2022-06-15 00:27:05 +12:00
Hamish Mackenzie 82bc945818 Fix ghc 9 windows cross compilation (#1450)
Fixes Windows cross compilation for GHC 9.0 and 9.2.

Updates wine to use the version that is included with the chosen Nixpkgs (it used to be pinned to an older version).

Moves the configuration of the GHC source into the GHC derivation.  This should make adding Hadrian support easier.

Propagates library dependences (not just pkgconfig ones) on windows so that any DLLs in those libraries can be copied for TH evaluation and to the `/bin` directory of executable components.

Adds gcc and mfcgthreads as library dependencies on Windows so that the DLLs they include will be found.

Use `$pkgsHostTarget` (instead of `ghc-pkg`) to find all the DLLs can copy them to the `/bin` directory of executable components.

Adds support for __int128_t and __uint128_t to language-c to fix aarch64-darwin builds.

Fixed reinstalling packages that come with patched versions in ghcjs.
2022-05-19 20:21:16 +12:00
Hamish Mackenzie 9aed9c3faf Update materialization (#1449) 2022-04-21 21:12:51 +12:00
DavHau 1ca99326b2 always clean the source, no matter what (#1403)
* always clean the source, no matter what

* update hydra
2022-03-19 02:08:11 +13:00
Hamish Mackenzie fa2d8a9b6f Add ghc 9.2.2 (#1394) 2022-03-08 14:34:04 +13:00
Hamish Mackenzie 41682ea1fd update pins (#1371) 2022-02-15 11:11:34 +13:00
Hamish Mackenzie 6101c6ea7b Add GHC 9.0.2 (#1338)
Co-authored-by: Ollie Charles <ollie@ocharles.org.uk>
2022-01-09 03:10:05 +13:00
Hamish Mackenzie 942fa5defb Fix check-hydra test (#1312)
* Fix check-hydra test

* Fix for ghc 9.2.1
2021-12-13 03:16:35 +13:00
Hamish Mackenzie 24b421cbfc Add ghc 9.2.1 and update internal cabal to 3.6.2 (#1286) 2021-11-13 12:43:04 +13:00
Hamish Mackenzie e7808afc97 Fix materialization for ghc901 (#1228) 2021-09-03 16:20:26 +12:00
Hamish Mackenzie 1f4c2c9b08 Add GHC 8.10.7 (#1220) 2021-09-01 22:43:31 +12:00
Hamish Mackenzie 58dff9550b Update some older GHC materialization files (#1222) 2021-09-01 20:21:30 +12:00
Teo Camarasu f624ca5662 Add GHC-8.10.6 (#1214)
Add GHC-8.10.6

Co-authored-by: Peter Becich <peter@simspace.com>
Co-authored-by: Teo Camarasu <teofilcamarasu@gmail.com>
Co-authored-by: Hamish Mackenzie <Hamish.K.Mackenzie@gmail.com>
2021-08-31 04:19:40 +12:00
Hamish Mackenzie 9cb9fd641c Fix buildkite job updating of flake.lock (#1193) 2021-08-10 12:35:38 +12:00
Hamish Mackenzie d5d304d5b5 Flakify all inputs (#1180)
* Include dependencies of haskell.nix that were tracked in `nix/sources.json`
  as flake inputs (`flake.lock` replaces `nix/sources.json`).
* Use `flake-compat` to continue to provide a compatible interface for non
  flake projects.

Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
Co-authored-by: Shea Levy <shea@shealevy.com>
2021-08-06 15:54:39 +12:00
Hamish Mackenzie f899936d1f Update internalHackageIndexState (#1139)
The use of `sources.HTTP` to work around issues building GHC 9.0.1 causes a mismatch between `flake` based and non
 `flake` based derivations.

The fixed version of HTTP is now in hackage so updating the internal hackage index state used haskell.nix removes the need for the HTTP source override.
2021-06-16 22:10:42 +12:00
Hamish Mackenzie a5b74bf002 niv update nixpkgs-unstable and add ghc 8.10.5 (#1121) 2021-06-15 19:50:45 +12:00
Hamish Mackenzie 5c25ce919f Add getComponent to project and package (#1060)
* Delays cabal configure errors until after evaluation when possible.

* Avoids rerunning long `cabal configure` if we already know it will fail.

* Provides a way to get a component from a package or project using a `cabal` like reference.

Code using the `tool` functions will automatically use `getComponent`.

For `(hackage-package {...}).components.library` is also ok.

```
# Consider changing hackage-package use:
(pkgs.haskell-nix.hackage-package {...}).components.exes.something
(pkgs.haskell-nix.hackage-package {...}).getComponent "exe:something"

# For any cabal project:
project.hsPkgs.somepackage.components.exes.something
project.getComponent "somepackage:exe:something"
# or do it in two steps
(project.getPackage "somepackage").getComponent "exe:something"
```

The reason for the new function is that we cannot provide the attribute interface without knowing that packages are in the project first.

Here is how the `cabal configure` error output is handled:

* The `plan-nix` derivation builds even if `cabal configure` fails.

* When it fails, it copies `failed-cabal-configure.nix` to the `$out/default.nix` along with a copy of the `cabal configure` output.

* When `failed-cabal-configure.nix` is imported and used in any way it writes the `cabal configure` output with `__trace` so it will always be visible.

* Instead of a `plan` the imported nix contains a `configurationError` pointing the `cabal configure` output.

* The intermediate functions `configurationError` and bubble it up to where it is needed.

* `getPackage` returns a mostly empty proxy for a real package when there is a `configurationError`

* The `getComponent` function always returns a derivation, but the version in the proxy writes the `cabal configure` output to stdout and calls `exit 1` (so that it will never build).
2021-03-04 20:10:58 +13:00
Hamish Mackenzie 413adb037f Update core interface files GHC to 8.10.4 (#1039) 2021-02-16 23:15:55 +13:00
Hamish Mackenzie 89f50a96dd Use pkgs.lib instead of stdenv.lib (#1031)
Fixes:

Warning: `stdenv.lib` is deprecated and will be removed in the next release. Please use `pkgs.lib` instead. For more information see https://github.com/NixOS/nixpkgs/issues/108938
2021-02-11 01:07:16 +13:00
Hamish Mackenzie b20a2dd1a3 Add missing materialization for 8.10 windows cross (#1002) 2021-01-18 19:16:35 +13:00
John A. Lotoski 09526c8555 Spelling, typo and whitespace fixes (#833)
* Spelling and typo fixes in doc and code comments
* Trailing whitespace or whitespace only truncation
* readTheDocs formatting corrections
2020-08-31 11:08:25 +12:00
Hamish Mackenzie cacfba0b45 Replace ghc 8.8.3 with 8.8.4 (#766)
Also adds some tooling/docs for adding new ghc versions (see `docs/adding-new-ghc.md`)
2020-07-21 14:10:23 +12:00
Hamish Mackenzie 4cac8bd00f Remove internal deps on default ghc and stackage (#738)
Changes to the interface of haskell.nix (from the changelog.md file):

* Removed `sources.nixpkgs-default`, use `sources.nixpkgs` instead.
* Removed `./nixpkgs` directory, use  `(import ./. {}).sources`
  or `./nix/sources.nix` instead.
* Removes V1 interface for details on how to fix old code see:
    https://github.com/input-output-hk/haskell.nix/issues/709
* Removed defaultCompilerNixName.
* cabalProject, cabalProject', hackage-project and hackage-package
  now require a `compiler-nix-name` argument.
* `haskell-nix.tool` and `.tools` now require a `compiler-nix-name` argument.
  New functions `p.tool` and `p.tools` (where p is a project) do not.
  Like `shellFor { tools = ... }` they will use the compiler nix name
  from the project (including stack projects where it is derived from
  the resolver).
* `haskell-nix.alex` and `haskell-nix.happy` have been removed. Use
  `p.tool "alex" "3.2.5"` or `shellFor { tools = { alex = "3.2.5"; } }`.
* `haskell-nix.nix-tools` -> `haskell-nix.nix-tools.ghc883` (it includes
  the hpack exe now).
* `haskell-nix.cabal-install` -> 
  `p.tool "cabal" "3.2.0.0"` or `shellFor { tools = { cabal = "3.2.0.0"; } }`
* `haskell-nix.haskellNixRoots` -> `haskell-nix.roots ghc883` or `p.roots`

Other changes:

Adds hpack executable to the nix-tools derivations.

Adds a `cabal-hpack` test to make sure `hpack` works with
`cabalProject`.

Reduces the number of calls to `cabalProject` (particularly when
checking materialization), by giving internal tools a per-compiler
attribute.

Uses happy 1.19.12 when building newer ghc versions.

Updates cabal-install 3.2.0.0 to use the source from github that
is compatible with ghc 8.10.1.

Updates the docs for callCabalProjectToNix.

Adds a license mapping to fix a common warning.
2020-07-08 22:54:01 +12:00
Hamish Mackenzie 29a9b936eb Test cabalProject when src is a non store path (#752) 2020-07-07 21:13:27 +12:00
Domen Kožar 59cf05606e module: generate using markdown template (#693)
The main motivation is that each option now is a heading and
such a link!
2020-06-16 17:13:04 +08:00
Hamish Mackenzie 5a8b3cb5da Check that materialize does not limit concurrency (#699)
Adding materialize call to a derivation should not prevent nix form
building it concurrently with other derivations.

This test makes sure that is the case.
2020-06-16 21:02:37 +12:00
Hamish Mackenzie edc948e0e8 Update nixpkgs 20.03 (matches with iohk-nix) (#671)
* Use hydra-migration in check-hydra.nix

* mkdocs fix is now in nixpkgs 20.03 (so this drops our work around)

* Adds sanity checks for #660

* Add missing ${targetPrefix} to NIX_LDFLAGS

* Use eval system for callStackToNix.

* Improves pins.
2020-06-14 22:54:03 +12:00
Michael Peyton Jones 701c83fb43 Update nixpkgs pins to include 20.03 (#557)
* Update nixpkgs pins to include 20.03, wobble

- Add 20.03
- Remove 19.03 (and old 18.09 file)
- Update 19.09 to latest
- Update github pin to 20.03
- Remove workarounds for 19.03

* Mark openssl as allowedInsecure

* Force python3 for mkdocs
2020-04-29 02:49:36 +12:00
Moritz Angermann 28b8e2244a Update update-index-state-hashes.nix 2020-03-26 14:17:32 +08:00
Michael Peyton Jones b6a8a97249 Get the hackage index tarball at runtime, not eval time (#512)
* Get the hackage index tarball at runtime, not eval time
* Get rid of special case for update-hackage

Fixes #507.
2020-03-25 20:26:31 +08:00
Michael Peyton Jones 64700ae79e CI: share between ci.nix and release.nix (#504)
* Define release.nix in terms of ci.nix

Beef up ci.nix to handle the same things as release.nix, then implement
the one in terms of the other.

* Fix infinite loop in evaluating release.nix with high ifdLevel

* Add tests to cross-compile

* Remove obsolete fix

* Do generic platform filtering

* Undefined variable

* Extract ci-lib.nix

* More undefined variables

* Use filterAttrsOnlyRecursive to avoid forcing all the drv attributes

* Set ifdLevel 0 to see if it fixes hydra eval

* Turn off tests on aarch64 cross

* Ifd level 1

* Ifd level 2

* Ifd level 3

* Maybe it's required that's too big?

Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
2020-03-25 11:20:58 +00:00
Moritz Angermann 2232e06112 bump scripts 2020-03-06 21:52:05 +08:00
Hamish Mackenzie 8e52ad6f96 Automatically generate cache for stackage projects (#397)
You can add a `# nix-sha256` comment to stack.yaml if you
need it to work in restricted mode.
2020-01-15 10:34:14 +13:00