Files
haskell.nix/docs/reference/library.md
T
Hamish Mackenzie 0366537651 Use new cleanSourceWith to filter component source (#209)
When trying to build haskell packages from local source using nix (
source code you have used `git clone` to retrieve and have modified)
it is often annoying that small changes can trigger a lot of components
to be built unnecessarily.  For instance if you change the code for
a test in a package it will often trigger builds of the library and
all its dependencies.

To avoid this problem we could manually set up `cleanSourceWith` calls
in nix to filter out files that are not need, but to do this every
component would be difficult and error prone.

This change automates this process using the information in the
packages `.cabal` file to filter the source of each component
automatically  when using `callCabalPlanToNix` (we should be able to
add it to `callStackToNix` too).

## How does it work?

* A new version on `cleanSourceWith` is used that adds a `subDir`
  argument.  It allows descending into a subdirectory to compose with
  cleaning the source.

* A new `--full` argument is passed to `plan-to-nix` by
  `callCabalPlanToNix` telling it to output extra information about
  the cabal packages in the plan (directories and filenames).

* `callCabalProjectToNix` uses the new `subDir` argument of
  `cleanSourceWith` when setting the packageSrc.

* `comp-builder.nix` and `setup-builder.nix` use a new
  `cleanCabalComponent` function to further clean the packageSrc
  using the new information in the `package` and `component`.

## hpack wildcards
`package.yaml` files can include wild cards and these are not handled
yet (they do not map to anything in the `.cabal` file and so are lost).
They will need to be added manually to the appropriate field
(eg. `packages.xyz.package.dataFiles`)

If your hpack file does not contain wildcards or if you are willing to
add the wildcards entries manually in nix it is still useful to
enable cleaning on hpack.

For example:

```
packages.ghc-toolkit.package.cleanHpack = true;
packages.ghc-toolkit.components.library.extraSrcFiles = [
  "genapply/**/**"
  "boot-libs/**/**"
  "ghc-libdir/**/**"
  ];
```
2019-08-26 12:27:47 +02:00

9.4 KiB
Raw Blame History

Haskell.nix contains a library of functions for creating buildable package sets from their Nix expression descriptions. The library is what you get when importing Haskell.nix. It might be helpful to load the library in the Nix REPL to test things.

Types

Package Set

The result of mkPkgSet. This is an application of the NixOS module system.

{
  options = { ... };
  config = {
    hsPkgs = { ... };
    packages = { ... };
    compiler = {
      version = "X.Y.Z";
      nix-name = "ghcXYZ";
      packages = { ... };
    };
  };
}
Attribute Type Description
options Module options The combination of all options set through the modules argument passed to mkPkgsSet.
config The result of evaluating and applying the options with Haskell.nix
.hsPkgs Attrset of Haskell Packages Buildable packages, created from packages
.packages Attrset of Haskell Package descriptions Configuration for each package in hsPkgs
.compiler Attrset

Haskell Package description

The Haskell package descriptions are values of the pkgSet.config.packages attrset. These are not derivations, but just the configuration for building an individual package. The configuration options are described under packages.<name> in Module options.

Component description

The component descriptions are values of the pkgSet.config.packages.<package>.components attrset. These are not derivations, but just the configuration for building an individual component. The configuration options are described under packages.<name>.components.* in Module options.

Haskell Package

In Haskell.nix, a Haskell package is a derivation which has a components attribute. This derivation is actually just for the package Setup.hs script, and isn't very interesting. To actually use the package, look within the components structure.

components = {
  library = COMPONENT;
  exes = { NAME = COMPONENT; };
  tests = { NAME = COMPONENT; };
  benchmarks = { NAME = COMPONENT; };
  all = COMPONENT;
}

Component

In Haskell.nix, a component is a derivation corresponding to a Cabal component of a package.

Haskell.nix also defines a special all component, which is the union of all components in the package.

Identifier

A package identifier is an attrset pair of name and version.

Extras

Extras allow adding more packages to the package set. These will be functions taking a single parameter hackage. They should return an attrset of package descriptions.

Modules

Modules are the primary method of configuring building of the package set. They are either:

  1. an attrset containing option declarations, or
  2. a function that returns an attrset containing option declarations.

If using the function form of a module, the following named parameters will be passed to it:

Argument Type Description
haskellLib attrset The haskellLib utility functions.
pkgs The Nixpkgs collection.
pkgconfPkgs A mapping of cabal build-depends names to Nixpkgs packages. (TODO: more information about this)
buildModules
config
options

Top-level attributes

mkStackPkgSet

Creates a package set based on the pkgs.nix output of stack-to-nix.

mkStackPkgSet =
    { stack-pkgs, pkg-def-extras ? [], modules ? []}: ...
Argument Type Description
stack-pkgs import ./pkgs.nix — The imported file generated by stack‑to‑nix.
pkg‑def‑extras List of Extras For overriding the package set.
modules List of Modules For overriding the package set.

Return value: a pkgSet

mkCabalProjectPkgSet

Creates a package set based on the pkgs.nix output of plan-to-nix.

mkCabalProjectPkgSet =
    { plan-pkgs, pkg-def-extras ? [], modules ? []}: ...
Argument Type Description
plan-pkgs import ./pkgs.nix — The imported file generated by plan‑to‑nix.
pkg‑def‑extras List of Extras For overriding the package set.
modules List of Modules For overriding the package set.

Return value: a pkgSet

mkPkgSet

This is the base function used by both mkStackPkgSet and mkCabalProjectPkgSet.

Return value: a pkgSet

snapshots

This is an attrset of hsPkgs packages from Stackage.

haskellPackages

A hsPkgs package set, which is one of the recent LTS Haskell releases from snapshots.

The chosen LTS is updated occasionally in Haskell.nix, though a manual process.

nix-tools

A derivation containing the nix-tools command-line tools.

callStackToNix

Runs stack-to-nix and produces the output needed for importAndFilterProject.

Example:

  pkgSet = mkStackPkgSet {
    stack-pkgs = (importAndFilterProject (callStackToNix {
      src = ./.;
    })).pkgs;
    pkg-def-extras = [];
    modules = [];
  };

callCabalProjectToNix

Runs cabal new-configure and plan-to-nix and produces the output needed for importAndFilterProject.

Example:

  pkgSet = mkCabalProjectPkgSet {
    plan-pkgs = (importAndFilterProject (callCabalProjectToNix {
      index-state = "2019-04-30T00:00:00Z";
      src = ./.;
    })).pkgs;

importAndFilterProject

Imports from a derivation created by callStackToNix or callCabalProjectToNix.

The result is an attrset with the following values:

Attribute Type Description
pkgs attrset that can be passed to mkStackPkgSet (as stack-pkgs) or mkCabalProjectPkgSet (as plan-pkgs).
nix this can be built and cached so that the amount built in the evaluation phase is not too great (helps to avoid timeouts on Hydra).

hackage

stackage

fetchExternal

cleanSourceHaskell

haskellLib

Assorted functions for operating on Haskell.nix data. This is distinct from pkgs.haskell.lib in the current Nixpkgs Haskell Infrastructure.

collectComponents

Extracts a selection of components from a Haskell package set.

This can be used to filter out all test suites or benchmarks of your project, so that they can be built in Hydra.

collectComponents =
    group: packageSel: haskellPackages: ...
Argument Type Description
group String A sub-component type.
packageSel A function Package -> Bool A predicate to filter packages with.
haskellPackages Package set All packages in the build.

Return value: a recursive attrset mapping package names → component names → components.

Example:

tests = collectComponents "tests" (package: package.identifier.name == "mypackage") hsPkgs;

Will result in moving derivations from hsPkgs.mypackage.components.tests.unit-tests to tests.mypackage.unit-tests.

subComponentTypes

Sub-component types identify components and are one of:

  • sublibs
  • foreignlibs
  • exes
  • tests
  • benchmarks

Package-set functions

These functions exist within the hsPkgs package set.

shellFor

Create a nix-shell development environment for developing one or more packages.

shellFor =
    { packages, withHoogle ? true, ...}: ...
Argument Type Description
packages Function Package selection function. It takes a list of Haskell packages and returns a subset of these packages.
withHoogle Boolean Whether to build a Hoogle documentation index and provide the hoogle command.
{ ... } Attrset All the other arguments are passed to mkDerivation.

Return value: a derivation

ghcWithPackages

Creates a nix-shell development environment including the given packages selected from this package set.

Parameter: a package selection function.

Return value: a derivation

Example:

haskell.haskellPackages.ghcWithPackages (ps: with ps; [ lens conduit ])

ghcWithHoogle

The same as ghcWithPackages, except, a hoogle command with a Hoogle documentation index of the packages will be included in the shell.