OpenLDAP invalid credentials immediately after setting credentials

I am having trouble binding users that are not the root dn in OpenLDAP, even if I immediately set the password, I still get ldap_bind: Invalid credentials (49)

For example, if I use ldappasswd to set the password (authenticating with the root dn), and then immediately use ldapwhoami to try to authenticate, I get the following error:

leif@nixos ~ $ ldappasswd -x -D cn=Admin,dc=leifandersen,dc=net -W -s badpasswd cn=leiftest,ou=users,dc=leif,dc=net
Enter LDAP Password: 
leif@nixos ~ $ ldapwhoami -x -w badpasswd -D cn=leiftest,ou=users,dc=leifandersen,dc=pl
ldap_bind: Invalid credentials (49)

(Note that in this case slapd is running on localhost and port 389, so I don't seem to need to specify those.)

I am using ppolicy, configured as:

# ppolicy, leifandersen.net
dn: cn=ppolicy,dc=leifandersen,dc=net
objectClass: device
objectClass: pwdPolicyChecker
objectClass: pwdPolicy
cn: ppolicy
pwdAllowUserChange: TRUE
pwdAttribute: userPassword
pwdCheckQuality: 1
pwdExpireWarning: 600
pwdFailureCountInterval: 30
pwdGraceAuthNLimit: 5
pwdInHistory: 5
pwdMaxAge: 0
pwdMaxFailure: 5
pwdMinAge: 0
pwdMinLength: 5
pwdMustChange: FALSE
pwdSafeModify: FALSE
pwdLockoutDuration: 30
pwdLockout: FALSE

And set up my initial configuration of OpenLDAP with nix, the relevant bit (I think) being:

"olcOverlay=ppolicy" = {
  attrs = {
    objectClass = [ "olcOverlayConfig" "olcPPolicyConfig" ];
    olcOverlay = "ppolicy";
    olcPPolicyDefault = "cn=ppolicy,dc=leif,dc=pl";
    olcPPolicyUseLockout = "FALSE";
    olcPPolicyHashCleartext = "TRUE";
  };
};

the whole config being:

services.openldap = {
  enable = true;
  settings = {
    attrs.olcLogLevel = [ "stats" ];
    children = {
      "cn=schema".includes = [
        "${pkgs.openldap}/etc/schema/core.ldif"
        "${pkgs.openldap}/etc/schema/cosine.ldif"
        "${pkgs.openldap}/etc/schema/inetorgperson.ldif"
        "${pkgs.openldap}/etc/schema/nis.ldif"
        "${pkgs.openldap}/etc/schema/ppolicy.ldif"
      ];
      "olcDatabase={-1}frontend" = {
        attrs = {
          objectClass = "olcDatabaseConfig";
          olcDatabase = "{-1}frontend";
          olcAccess = [ "{0}to * by dn.exact=uidNumber=0+gidNumber=0,cn=peercred,cn=external,cn=auth manage stop by * none stop" ];
        };
      };
      "olcDatabase={0}config" = {
        attrs = {
          objectClass = "olcDatabaseConfig";
          olcDatabase = "{0}config";
          olcAccess = [ "{0}to * by * none break" ];
        };
      };
      "olcDatabase={1}mdb" = {
        attrs = {
          objectClass = ["olcDatabaseConfig" "olcMdbConfig"];
          olcDatabase = "{1}mdb";
          olcDbDirectory = "/var/db/ldap";
          olcDbIndex = [
          "objectClass eq"
            "cn pres,eq"
            "uid pres,eq"
            "sn pres,eq,subany"
          ];
          olcSuffix = "dc=leifandersen,dc=net";
          olcAccess = [ "{0}to * by * none break" ]; # read break for readable
          olcRootDN = "cn=Admin,dc=leifandersen,dc=net";
          olcRootPW = "{SSHA}<SOMEHASH>";
        };
        children = {
          "olcOverlay=ppolicy" = {
            attrs = {
              objectClass = [ "olcOverlayConfig" "olcPPolicyConfig" ];
              olcOverlay = "ppolicy";
              olcPPolicyDefault = "cn=ppolicy,dc=leif,dc=pl";
              olcPPolicyUseLockout = "FALSE";
              olcPPolicyHashCleartext = "TRUE";
            };
          };
        };
      };
    };
  };
};

Does anyone have any idea why I'm getting an invalid credentials error? Is there something wrong with my ppolicy setup? (I know its not as strict as it should be, I laxed the requirements a bit in the hopes I could get something to work. Also sorry if this is a bad question, I'm still very new to ldap and my previous searches didn't turn up any answer.)

OpenLDAP requires during authentication binds that access be explicitly granted for the necessary attributes (particularly userPassword) to be used during authentication by the state of the bind before authentication (i.e. anonymous). Try changing the line

          olcAccess = [ "{0}to * by * none break" ]; # read break for readable

to

          olcAccess = [
            "{0}to attr=userPassword by anonymous auth"
            "{1}to * by * none break"
          ]; # read break for readable

in the "{1}mdb" section, as suggested here.