Using FsCheck I get different results on tests, once 100% passed and the other time error

I created a generator to generate lists of int with the same lenght and to test the property of zip and unzip. Running the test I get once in a while the error

Error: System.ArgumentException: list2 is 1 element shorter than list1

but it shouldn't happen because of my generator.

I got three times the test 100% passed and then the error above. Why? It seems my generator is not working properly.

let samelength (x, y) = 
    List.length x = List.length y

let arbMyGen2 = Arb.filter samelength Arb.from<int list * int list> 

type MyGenZ =
    static member genZip() = 
       {
        new Arbitrary<int list * int list>() with
            override x.Generator = arbMyGen2 |> Arb.toGen
            override x.Shrinker t = Seq.empty
    }

let _ = Arb.register<MyGenZ>()

let pro_zip (xs: int list, ys: int list) = 
   (xs, ys) = List.unzip(List.zip xs ys)
   |> Prop.collect (List.length xs = List.length ys)

do Check.Quick pro_zip

Solution

Your code, as written, works for me. So I'm not sure what exactly is wrong, but I can give you a few helpful (hopefully!) hints.

In the first instance, try not using the registrating mechanism, but instead using Prop.forAll, as follows:

let pro_zip  = 
   Prop.forAll arbMyGen2 (fun (xs,ys) ->
        (xs, ys) = List.unzip(List.zip xs ys)
        |> Prop.collect (List.length xs))

do Check.Quick pro_zip

Note I've also changed your Prop.collect call to collect the length of the list(s), which gives somewhat more interesting output. In fact your property already checks that the lists are the same length (albeit implicitly) so the test will fail with a counterexample if they are not.

Arb.filter transforms an existing Arbitrary (i.e. generator and filter) to a new Arbitrary. In other words, arbMyGen2 has a shrinking function that'll work (i.e. only returns smaller pairs of lists that are of equal length), while in genZip() you throw the shrinker away. It would be fine to simply write

type MyGenZ =
    static member genZip() = arbMyGen2

instead.