Skip to content

hosts/borg: Auto-update to avoid invalid NIX_PATH

André Breda requested to merge ist189409/nixrnl:borg-update into master

Description of changes

Labs machines update automatically every night. This is great, but means that sometimes borg is on a different nixpkgs revision.

Problem: Slurm sends environment variables to cluster nodes, meaning NIX_PATH can point to a different nixpkgs revision, potentially already garbage collected in lab machines.

This MR:

  • Imports the pull profile in borg to update it automatically.
    • This will diminish the chance of it sending an invalid NIX_PATH.
    • (bonus) This diminishes the chance of discrepancies between borg and cluster nodes Slurm config.
  • (bonus) Changes the pull profile repo URI to point to RNL's GitLab (it was pointing to the GitHub mirror for historical reasons that no longer make sense) Meanwhile this was done in parallel.

Limitations

NIX_PATH can still be invalid, e.g. when a borg user has a long-running tmux: the tmux and its subprocesses will keep the old paths, which is fine in borg, nix will not remove them, but they may still be removed in cluster nodes. The problem also applies to other environment variables (SLURM_CONF, SHELL, LESSOPEN, LESSKEYIN_SYSTEM from a quick srun env | grep nix), which may warrant a more elaborate solution (perhaps removing them automatically and re-setting a few of them to good values in each task/node?).

I suggest opening a separate issue to track further improvement.

Things done

  • Tested
  • Updated documentation (Wiki/NetBox)
  • Breaking change
Edited by André Breda

Merge request reports

Loading