Hidden packages in Haskell again. Is the documentation up to date?

I am getting

*Could not load module ‘Data.Array’
It is a member of the hidden package ‘array-0.5.4.0’.
You can run ‘:set -package array’ to expose it.
...\[snip\]...*

This when compiling Simon Marlow's code from the parallel book. $ ghc -O2 sudoku1.hs -rtsopts

Of course, there have been many similar questions asked here, and I have skimmed all the answers, and read the ghc documentation on packages and more.

I can, of course fix this with -package array . But this gets ridiculously cumbersome in later programs where I need many -package switches.

My installation is via ghcup and I have the recommended GHC 9.4.8, cabal 3.12.1.0 & stack 3.3.1. For now I want to work with GHC directly and understand what is happening.

I have ~/.ghcup/ghc/... , ~/.ghc/... and ~/.cabal/ the first and the last have package files.

However, the documentation suggests that ghc-pkg list should see all of the "registered" packages, but it seems to omit some in the cabal area. Leaving that aside for now, ghc-pkg does list array.

Further more ghc-pkg says that array is exposed :-

$ ghc-pkg describe array

name:                 array
version:              0.5.4.0
visibility:           public
id:                   array-0.5.4.0
key:                  array-0.5.4.0
license:              BSD-3-Clause
maintainer:           libraries@haskell.org
synopsis:             Mutable and immutable arrays
description:
In addition to providing the "Data.Array" module
\<http://www.haskell.org/onlinereport/haskell2010/haskellch14.html as specified in the Haskell 2010 Language Report\>,
this package also defines the classes 'IArray' of
immutable arrays and 'MArray' of arrays mutable within appropriate
monads, as well as some instances of these classes.

category:             Data Structures
abi:                  41bd88e1446deebe13fe286fbd046e2f
exposed:              True    **\<====================**

..\[snip\]..

So why isn't ghc importing Data.array without a -package switch?

Clearly I am pretty confused about what is going on, and the documentation in the ghc user's guide is wrong or I am missing something.

Solution

I'm going to annoy @DanielWagner by jumping on his answer and adding a ton of detail. Anyway, he's right: GHC is loading a package environment, probably a default environment located at:

~/.ghc/x86_64-linux-9.4.8/environments/default

You can find out for sure by running ghc -v, and checking the first line:

buhr@delores:~$ ghc -v
Loaded package environment from /u/buhr/.ghc/x86_64-linux-9.4.8/environments/default

In a fresh GHCup installation, this file does not exist, and so no environment is loaded. As a result, GHC follows its default package database rules and makes available all exposed packages in:

the "global package database". This is located in /usr/lib/ghc-x.x.x/package.conf.d for traditional GHC installs but in ~/.ghcup/x.x.x/lib/ghc-x.x.x/lib/package.conf.d for GHCup-managed installs.
the "user package database", located in ~/.ghc/x86_64-linux-x.x.x/package.conf.d. By default, this doesn't exist but can be created and manipulated via ghc-pkg commands. For example, ghc-pkg recache --user will create an empty user package database which will then appear in addition to the global database in the output of ghc-pkg list. These days, I don't think anyone really uses the "user package database", but see below.

However, if you, at any time, use Cabal's --lib argument to globally install a package, like cabal install --lib acme-missles, then Cabal will barf out a warning that you will probably ignore:

Warning: The libraries were installed by creating a global GHC environment
file at:
/u/buhr/.ghc/x86_64-linux-9.4.8/environments/default

The presence of such an environment file is likely to confuse or break other
tools because it changes GHC's behaviour: it changes the default package set
in ghc and ghci from its normal value (which is "all boot libraries"). GHC
environment files are little-used and often not tested for.

Furthermore, management of these environment files is still more difficult
than it could be; see e.g. https://github.com/haskell/cabal/issues/6481 .

Double-check that creating a global GHC environment file is really what you
wanted! You can limit the effects of the environment file by creating it in a
specific directory using the --package-env flag. For example, use:

cabal install --lib <packages...> --package-env .

to create the file in the current directory.

and will create a default environment file that looks something like this:

clear-package-db
global-package-db
package-db /u/buhr/.cabal/store/ghc-9.4.8/package.db
package-id base-4.17.2.1
package-id acme-missiles-0.3-1e6ff7f7f156e1dd63e8acdd09d4814b43d47bfdaf9bfd2653eacebdf90414f0

These directives act like the corresponding command line arguments for every GHC (and GHCi) invocation that uses this default environment, with an additional -hide-all-packages argument implicitly assumed. The result is that all packages are hidden by default, regardless of their ghc-pkg-reported exposure status, the "user package database" under ~/.ghc is dropped from the search list in favor of the indicated ~/.cabal database, and only base and cabal install --lib ...-installed packages are exposed by default, with any other packages requiring a -package argument.

If, like me, you don't much care about the state of ~/.ghcup, ~/.ghc, ~/.cabal, and ~/.stack and are always ready and willing to clear them out and reinstall GHCup from scratch, then a simple solution is to continue to use cabal install --lib ... to install packages and add them to the default environment file. This will make them available as exposed packages when working "globally" outside of any Cabal or Stack project. Things might get wedged if a set of incompatible packages gets installed, and that's where clearing everything out and reinstalling comes into play.

Alternatively, you can go old school and install packages in the user package database. Remove the default environment file to revert GHC to its default behavior. ghc-pkg list will show all the available packages (e.g., installed in the global database). If you want to add a package to your user database, you can fetch, build, and register it using legacy Cabal commands:

$ cabal get acme-missiles
Unpacking to acme-missiles-0.3/
$ cd acme-missiles-0.3/
$ cabal v1-build
Resolving dependencies...
Configuring acme-missiles-0.3...
Preprocessing library for acme-missiles-0.3...
Building library for acme-missiles-0.3...
[1 of 2] Compiling Acme.Missiles    ( Acme/Missiles.hs, dist/build/Acme/Missiles.o, dist/build/Acme/Missiles.dyn_o )
[2 of 2] Compiling Acme.Missiles.STM ( Acme/Missiles/STM.hs, dist/build/Acme/Missiles/STM.o, dist/build/Acme/Missiles/STM.dyn_o )
$ cabal v1-register --user
Registering library for acme-missiles-0.3...
$ ghc-pkg list
...
/u/buhr/.ghc/x86_64-linux-9.4.8/package.conf.d  <-- my user package database
    acme-missiles-0.3
$ cd ..  # get out of the Cabal-controlled project directory
$ ghci   # invoke GHCi outside of Cabal control
GHCi, version 9.4.8: https://www.haskell.org/ghc/  :? for help
ghci> import Acme.Missiles   -- the registered package is used here
ghci> launchMissiles
Nuclear launch detected.
ghci>

Again, I'm not sure anyone does this anymore, but it's a possible approach.

The recommended approach is the one that's most inconvenient for "casual" GHC programming, and that's putting all of your work, even one-off scripts and tests, into a proper Cabal or Stack project, and using the per-project environment they automatically set up. But, you probably know all that...