test runner: concurrency boolean flag

[MoLow]

following up the discussion in #43757
if we want to allow the concurrency option to be a boolean, we need to come up with good defaults

there are two levels of concurrency:

1. when running multiple test files, using --test each file is spawned in its own process so we use os.cpus().length as the default
2. inside a test file - we use 1 as the default wich will be equivalent to false

Assuming we want to allow passing concurrency: true in a test file - what number would that convert to?

[MoLow]

cc @nodejs/test_runner @lpinca @JakobJingleheimer

[ljharb]

We should never use os.cpus().length as a default; we should use that minus 1.

[JakobJingleheimer]

Actually, we need to spawn a process for each case (it()s)? ex

describe('example', { concurrency: true } () => {
  it('should add A and B', () => {
    assert.strictEqual(arithmetic(1,2, 'add'), 3);
  });

  it('should subtract A from B', () => {
    assert.strictEqual(arithmetic(1,3, 'subtract'), 2);
    assert.strictEqual(arithmetic(3,1, 'subtract'), -2);
  });
});

Those 2 it()s should be run in parallel, but they don't need isolated processes. In that case, do we care about os.cpus() at all?

[juliangruber]

Concurrent it tests will run in the same process, you only leverage more cpu cores by splitting your tests across multiple files

[JakobJingleheimer]

Soooo, why is concurrency an integer at all?

[aduh95]

Concurrent it tests will run in the same process, you only leverage more cpu cores by splitting your tests across multiple files

True, except if those it tests spawn new thread/processes on their own.

[juliangruber]

Integer concurrency on the file level lets you control how many processes will be spawned, integer concurrency on the test level lets you limit how much work is done in parallel inside a single process

[JakobJingleheimer]

integer concurrency on the test level lets you limit how much work is done in parallel inside a single process

I can certainly understand wanting to limit the number of process spawned, but why would anyone care to limit the number of cases within that process to a specific number?

[juliangruber]

Say you have a large file with 2000 it() tests. Some perform compute-heavy sync work, others use resources like the file system, which doesn't provide unlimited file descriptors. It can be necessary or beneficial then to limit how many tests run at the same time.

[JakobJingleheimer]

Ahh, okay. Seems a bit of an edge-case; I wouldn't expect an edge-case config option to be the default, but good to have supported.

[JakobJingleheimer]

Assuming we want to allow passing concurrency: true in a test file - what number would that convert to?

Given that it's not related to cpus, does it need to convert to a number?