Stryker JS has started to generate useless reports after adding a large number of unit tests

[vladyslav-oleshko]

Disclaimer.
I am a front-end developer in a commercial project. Because of this, I can't share the code except some very short snippets (after anonymization). I don't expect understanding and just hope you could help.

Question
We've started a massive unit testing campaign on the project and decided to use your efficiency tool to reach maximum quality of the testing. For a while, everything was fine. But after creating about 8000 unit tests, the results of the Stryker report began to surprise me more and more. Two different runs give two different results. Moreover, components that had been covered long ago with 100% efficiency and had not been touched since then were shown in the new results as full of undetected mutations. As new unit tests were added, things got even worse. After creating 9000 unit tests, we simply stopped using the tool, even though we really liked it.

I have tried using newer version of Stryker, different configurations based on my understanding of documentation, limiting scope of mutation, etc. Nothing has helped. I have seen some prominent results after physical removing of tests, but that is not the way.

Are you familiar with the described issue? Do you have any advices that could help?
I'm able to verify them one by one.

Stryker environment

As I said, I can't provide all the code, but still that might be enough:

stryker.conf.json:

{ "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json", "mutate": [ // NDA. But I can assure that it includes only .ts files I need to mutate and exclude all secondary and .spec files ], "testRunner": "karma", "karma": { "configFile": "karma.conf.js", "projectType": "angular-cli", "config": { "browsers": [ "ChromeHeadless" ] } }, "timeoutMS": 60000, // we have a lot of async unit tests and that's the fix "concurrency": 8, // but I've tried all possible combination even without it at all (n-1 as mentioned in the specs) "coverageAnalysis": "perTest", "checkers": ["typescript"], "tsconfigFile": "tsconfig.json" }

The only plugin is "typescript". Also tried to use "mutator": { name: 'typescript' } both with checker and without it.

Current Stryker versions:
"@stryker-mutator/core": "^7.1.1",
"@stryker-mutator/karma-runner": "^7.1.1",
"@stryker-mutator/typescript-checker": "^7.1.1",

Tried the latest one at the moment of ticket creating which is 8.7.0. No changes in results have been noticed.

Other versions:
"karma": "~6.4.2"
"@angular/cli": "16.2.2",

p.s.
Even if you can't help me, I'd like to thank you for the amazing tool you've created.