How to get immediate output from a job run within gitlab-runner?

The command gitlab-runner lets you "test" a gitlab job locally. However, the local run of a job seems to have the same problem as a gitlab job run in gitlab CI: The output is not immediate!

What I mean: Even if your code/test/whatever produces printed output, it is not shown immediately in your log or console.

Here is how you can reproduce this behavior (on Linux):

Create a new git repository
```
 mkdir testrepo
 cd testrepo
 git init
```

Create file .gitlab-ci.yml with the following content

  job_test:
   image: python:3.8-buster
   script:
     - python tester.py

Create a file tester.py with the following content:

 import time

 for index in range(10):
     print(f"{time.time()}  test output")
     time.sleep(1)

Run this code locally
```
 python tester.py
```

which produces the output

1648130393.143866  test output
1648130394.1441162  test output
1648130395.14529  test output
1648130396.1466148  test output
1648130397.147796  test output
1648130398.148115  test output
1648130399.148294  test output
1648130400.1494567  test output
1648130401.1506176  test output
1648130402.1508648  test output

with each line appearing on the console every second.

You commit the changes

 git add tester.py
 git add .gitlab-ci.yml
 git commit -m "just a test"

You start the job within a gitlab runner

 gitlab-runner exec docker  job_test

 ....
 1648130501.9057398  test output
 1648130502.9068272  test output
 1648130503.9079702  test output
 1648130504.9090931  test output
 1648130505.910158  test output
 1648130506.9112566  test output
 1648130507.9120533  test output
 1648130508.9131665  test output
 1648130509.9142723  test output
 1648130510.9154003  test output
 Job succeeded

Here you get essentially the same output, but you have to wait for 10 seconds and then you get the complete output at once!

What I want is to see the output as it happens. So like one line every second.

How can I do that for both, the local gitlab-runner and the gitlab CI?

Solution

In the source code, this is controlled mostly by the clientJobTrace's updateInterval and forceSendInterval properties.

These properties are not user-configurable. In order to change this functionality, you would have to patch the source code for the GitLab Runner and compile it yourself.

The parameters for the job trace are passed from the newJobTrace function and their defaults (where you would need to alter the source) are defined here.

Also note that the UI for GitLab may not necessarily get the trace in realtime, either. So, even if the runner has sent the trace to GitLab, the javascript responsible for updating the UI only polls for trace data every ~4 or 5 seconds.