Suppose you have a repository with a submodule checked out in the
subdirectory `smod`.
`cleanGit { src = ./smod; }` would return the following error:
error: getting status of '/.git/modules/smod': No such file or directory
`smod/.git` is a file that contains a reference to the git module repo.
gitdir: ../.git/modules/smod
This is the same as a worktree (in fact, isWorktree = true here),
but it's a relative path instead of an absolute path.
Co-authored-by: Ewout Van Troostenberghe <evantroostenberghe@hubspot.com>
`runCommandLocal` sets both `preferLocalBuild=true` and
`allowSubstitutes=false`. There's an
[argument](https://github.com/NixOS/nix/issues/4442) that the latter is a
misfeature. There's really no reason not to download something from a
substitutor if it's actually been built already.
We can get the effect that we want easily enough by just setting
`preferLocalBuild=true` alone.
Currently in order to support submodules all the directories are included (but not all files) when building the whitelist. This is done because we need the `.git` files (or directories) the contain to be present when we run `git ls-files --recurse-submodules`.
Including every directory causes two performance issues:
* Creating a directory in your repo (even empty one not added to git at all) will require the whitelist to be recalculated.
* Recalculation of the whiltelist may be slowed down if there are a large number of directories. In particular `lstat` is called on every file (probably to get the right `type` to pass to the filter function).
The fix is to use `git submodule status --recursive` to get a list of all the submodule directories. Then the `.git` files (or directories) in them can be included. This has to be repeated until all the submodules are known (as `--recursive` will not see the submodules until the parent module is included).
* Adds support for cross package refs (with a project). Relative
directory references between packages within a project should now
work.
* Adds `includeSiblings` to `cleanSourceWith`. When `true` it
prevents the `subDir` arg from causing filtering of other directories.
* Adds `keepGitDir` to `cleanGit` to allow `.git` directory to be kept
(useful for components that use the `githash` package).
```
cleanGit {
src = cleanSrcWith { ... }
}
```
Results in the full `src` being copied to the store before filtering
the files not in the git index.
This change uses `src.origSrcSubDir` if it exists to avoid this.
Some times it is handy to temporarily use a relative path between git
repos. If the repos are individually cleaned this is not possible
(since the cleaned version of one repo will never include the files
of the other).
`cleanGits` allows us to specify a root directory and any number of
sub directories containing git repos.
When no name is provided to `cleanSourceWith` or `cleanGit` we
currently use `baseNameOf src` as a default.
This was cute, but it lead to cache misses. For instance if
`x = cleanSourceWith { src = ./.; }` then `baseName src`
will be different when `src` resolves to "/nix/store/X"
than when it is in a local directory. If people use
git worktrees they also may wind up with different
values for `name`. Anything that depends on `x.name` will
propagate the issue.
This change uses `subDir` if present or if it is not it encourages
adding a suitable `name` with a warning message.
Repos that have submodules with submodules will have a `.git` file
that references the relative location of the submodules dir in the
`.git/modules`. Without this file the contents of submodules of
submodules are excluded from the `cleanGit` output.
When we are not in a worktree the submodule info is in the
`.git` dirs scattered throughout the repo itself. In that situation
we need to copy the `.git` dirs into the temp location we are using
to run `git ls-files`.
The Overlays branch
This is a major reorganization in how haskell.nix is used, and marks our 1.0 release.
The branch doesn't build due to numerous issues that we believe to be with the CI
and not this branch. We expect only very minor adjustments prior to calling this the
official 1.0 release.
* Move iohk-nix patches into haskell.nix
This moves the customizations we did in iohk-nix
into haskell.nix via overlays and config.
Add bootPkgs logic
this moves the nuking of bootPkgs where it belongs. This should eventually still be
removed and replaced by a proper solution, that doesn't require the nuking of
bootPkgs.
Allow us to bootstrap binary ghcs and a cabal-install
With this we can do the following:
```
$ nix repl compiler/old-ghc-nix
nix-repl> :b (let pkgs = import <nixpkgs> {}; in with import ./. {}; import ./compiler/bootstrap/cabal-install.nix { inherit (pkgs) fetchurl stdenv zlib; inherit hackage
; ghc = ghc844; src = pkgs.fetchurl { url = "https://github.com/haskell/cabal/archive/Cabal-v3.0.0.0-rc3.tar.gz"; sha256 = "1zl2mgg8307ykq3v8nmafc6zdhhj1cw7w8ffpap16dsm6
5lbnx33"; }; })
```
which wile it may look daunting, will allow us to bootstrap a
cabal-install with a ghc. From that point onwards, we should be able to
build any hackage package via haskell.nix.
Pass through cabal-install version
Better threading of arguments.
Add bootstrap overlay
Allow alex + happy to be built
This still has a wart: we need nix-tools, and for that we use the ghc865 from
nixpkgs. Ideally we'd build nix-tools against a specific ghc, but then
we'd need a build expression for that.
Make ghcjs work
Building something like this:
```
nix build '(with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; })).components.exes.hello'
```
will now work. Assuming `nixpkgs` has been appropriately patched to support the `js-unknown-ghcjs` triple.
Also: this will need an additional `Cabal` patch, to make `Cabal` understand what it needs to do with: `dist/build/hello/hello: copyFile: does not exist (No such file or directory)`
It needs to learn that `dist/build/hello/hello.jsexe` is what it wants to copy and that that is a directory. Luckily we do have some code in Cabal already that does this for `.exe` on windows.
Build `js-unknown-ghcjs` packages with haskell.nix
Using the following expression:
```
with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; };
let Cabal = buildPackages.haskell-nix.hackage-package {
name = "Cabal"; version = "2.4.1.0";
modules = [
{ packages.Cabal.patches = [ ./Cabal-install-folder.diff ]; }
];
}; in
(haskell-nix.hackage-package {
name = "hello"; version = "1.0.0.2";
modules = [
({config, ... }:{ packages.hello.package.setup-depends = [ Cabal ]; })
];}).components.exes.hello
```
in a `test.nix` file. And running
```
nix build -f ./test.nix
```
on it, will produce
```
./result
├── bin
│ └── hello.jsexe
│ ├── all.js
│ ├── all.js.externs
│ ├── index.html
│ ├── lib.js
│ ├── manifest.webapp
│ ├── out.frefs.js
│ ├── out.frefs.json
│ ├── out.js
│ ├── out.stats
│ ├── rts.js
│ └── runmain.js
└── share
└── doc
└── x86_64-linux-ghc-8.6.5
└── hello-1.0.0.2
└── LICENSE
6 directories, 12 files
```
Ewout
Michael Peyton Jones
Cheng Shao
Hamish Mackenzie
Thomas Tuegel
Alexander Bantyev
Samuel Leathers
Jonas Carpay
Moritz Angermann