Achieving partial build for broken Go codebase

I have a generator which generates me code in Go, for example:

• foo.go
• bar.go
• ...

After generation, I need to test if my generator is doing good job. For every generated file, I have a test counterpart:

• foo_test.go
• bar_test.go
• ...

Now, the problem is that there are hundreds of these tests, and given nature of the project, some of them will be sometimes broken — i.e. potentially "foo.go" might even fail to compile due to syntax error or something like that.

What I want to do is:

1. Attempt to compile everything
2. For files which fail to compile, capture compiler problems in a separate file and ignore them onwards for test purposes
3. For the rest of files (which are syntactically correct and compile successfully), along with their tests, I want to run the tests and capture test output

To make things more complicated, ideally I'd love to keep generates files (foo.go) separate from test files (foo_test.go) — both in separate directories and separate Go packages, something like that:

• generated/foo.go
• generated/bar.go
• tests/foo_test.go
• tests/bar_test.go

Keeping them in separate dir allows easy cleanups and separation of version control concerns. Keeping them in separate Go package allows testing that all interfaces I need to be generated are indeed public and would guarantee that tests would never touch private fields.

What I've tried so far

Single dir, single package

Algorithm:

• Put everything in one directory
• Have go.mod + go.sum with necessary dependencies and module declaration in there
• Have all files declare same package my_tests
• Start with go test -v foo.go foo_test.go bar.go bar_test.go
• If any of these fail (for example, "bar.go" has an error), capture compiler messages and retry without offending files: go test -v foo.go foo_test.go bar_test.go → and then eventually go test -v foo.go foo_test.go because bar_test.go won't survive without bar.go.

It kind of works, but the downsides are massive: I have to put everything in one directory (so git management becomes very messy) and I have to put everything in one Go package (which means that errors of test using private field occasionally might slip in).

Two modules

Algorithm:

• Two separate modules in separate dirs, each with their own go.mod + go.sum.

• First, I attempt to compile generated package and module with go build foo.go bar.go → bar.go is broken → go build foo.go succeeds at building a package

• Next, I go to tests package and module. That one has go.mod including replacement directive:

  replace my_generated => ../generated
  

  In it, I try to do go test -v foo_test.go — but, alas, this doesn't seem to work, as go test will make sure that full package "my_generated" is rebuilt, and that will ultimately fail, as previously I've only made it succeed with manually specifying list of allowed files.

Disable compilation offending files by deleting/renaming them

Same as previous, but instead of removing files from go build / go test invocation, hard disable go compiler from finding them to deleting or renaming them.

Works, but it's really ugly as it modifies test directory, so I either need to do full copy of it and run it in separate place, or rename .go -> .disabled and then rename it back after test run.

So, the question is: is there any better method/algorithm I can employ here? Maybe some non-obvious switch in go build system?

Maybe some non-obvious switch in go build system?

As mentioned in the comments, build constraints may provide an alternative to deleting files.

For example if foo.go and foo_test.go have the constraint //go:build !foo then using go test -tags foo would exclude those files (and you can specify multiple tags go test -tags foo,bar).

Note: This really just answers your specific question seeking another option. I cannot comment on whether this is really a good option (you question focuses on the what, and does not really mention the why).