Not able to remove EventListener in Typescript

I am trying to remove an eventlistener in typescript. I am adding eventlisteners in an if-statement. In the else-statement I am trying to remove these eventlisteners, but for some reason it does not remove them.

FYI: I have a button where I set a boolean (movePick). If it is true then I want to be able to Move my Object. Thats where the eventlisterners are created. If I click the button again I should not be able to move the object anymore. That's why I am trying to remove the eventlisteners.

public moveObject(movePick: boolean, raycaster: THREE.Raycaster): void {
        const plane = new THREE.Plane();
        const pNormal = new THREE.Vector3(0, 1, 0); // plane's normal
        const planeIntersect = new THREE.Vector3(); // point of intersection with the plane
        const pIntersect = new THREE.Vector3(); // point of intersection with an object (plane's point)
        const shift = new THREE.Vector3(); // distance between position of an object and points of intersection with the object
        let isDragging = false;
        let dragObject: any;
        // events

        const pointermove = (event: PointerEvent) => {
            const rect = this.canvas.getBoundingClientRect();
            const x = event.clientX - rect.left
            const y = event.clientY - rect.top
            const pickPosition = { x: 0, y: 0 };
            pickPosition.x = (x / this.canvas.clientWidth) * 2 - 1;
            pickPosition.y = (y / this.canvas.clientHeight) * -2 + 1;
            raycaster.setFromCamera(pickPosition, this.camera);
            // console.log("movePICK IN POINTERMOVE",movePick)
            if (isDragging) {
                raycaster.ray.intersectPlane(plane, planeIntersect);
                dragObject.position.addVectors(planeIntersect, shift);
            }
        }

        const mousedown = () => {
            const intersects = raycaster.intersectObjects(this.scene.children);

            for (let i = 0; i < this.mesharr.length; i++) {
                if (intersects[0].object.name == this.mesharr[i].name && intersects[0].object.name != "Rail") {
                    pIntersect.copy(intersects[0].point);
                    plane.setFromNormalAndCoplanarPoint(pNormal, pIntersect);
                    shift.subVectors(intersects[0].object.position, intersects[0].point);
                    isDragging = true;
                    dragObject = intersects[0].object;
                }
            }
        }

        const pointerup = () => {
            isDragging = false;
            dragObject = null;
        }
        **if (movePick) {
            document.addEventListener("pointermove", pointermove);
            document.addEventListener("mousedown", mousedown);
            document.addEventListener("pointerup", pointerup);
        } else {
            document.removeEventListener("pointermove", pointermove);
            document.removeEventListener("mousedown", mousedown);
            document.removeEventListener("pointerup", pointerup);
        }**
    }

If I remove the eventlisternes in the same if-statement that I added them in, they get removed. But if the button is clicked again and it enters the else-statement then they can not be removed. I also tried several solutions in stackoverflow but none of them would work.

Save the instances of the events to the class field and then call the instance in removeEventListener eg.

 document.removeEventListener("pointermove", this.savedpointermove); 

and now it should work