Allow adding parameter names to auto-generated parametrized test IDs

[Bastian-Krause]

By default, only the parameter's values make it into parametrized test IDs. The parameter names don't. Since parameter values do not always speak for themselves, the test function + test ID are often not descriptive/expressive.

Allowing parameter name=value pairs in the test ID optionally to get an idea what parameters a test gets passed is beneficial. So add a kwarg id_names to @pytest.mark.parametrize() / pytest.Metafunc.parametrize(). It defaults to False to keep the auto-generated ID as before. If set to True, the argument parameter=value pairs in the auto-generated test IDs are enabled. Calling parametrize() with ids and id_names=True is considered an error.

Auto-generated test ID with id_names=False (default behavior as before):

test_something[100-10-True-False-True]

Test ID with id_names=True:

test_something[speed_down=100-speed_up=10-foo=True-bar=False-baz=True]

Closes #13055

[dongfangtianyu]

Exciting proposal, really helpful for my current work!

Have you considered going further? Allow custom parameters for IDs (default is "all") to avoid losing readability when there are too many.

Inspired by: https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#output-template

For example: id_names='speed_down,baz'

test_something[speed_down=100-baz=True]

Or: id_template='${speed_down}-${speed_up}'

test_something[speed_down=100-speed_up=10]

[Bastian-Krause]

@dongfangtianyu If you ignore certain parameters, won't the IDs lose their uniqueness?

[Bastian-Krause]

The CI checks were green before rebasing and seem to happen on the main branch as well.

[dongfangtianyu]

If you ignore certain parameters, won't the IDs lose their uniqueness?

Yes, a simple example : non unique IDs will be automatically suffixed:

@pytest.mark.parametrize(
    "a",
    [6,6,6,9,9,9],
)
def test(a):
    pass

collected 6 items                                                                                       

test_a.py::test[6_0] PASSED
test_a.py::test[6_1] PASSED
test_a.py::test[6_2] PASSED
test_a.py::test[9_0] PASSED
test_a.py::test[9_1] PASSED
test_a.py::test[9_2] PASSED

[dongfangtianyu]

At first glance, it seems that there are two different functions supporting ids_name, and perhaps only @pytest.mark.parameterize() should be show. Additionally, it would be very enticing if you added an example in the changelog, such as: #12492.

[Bastian-Krause]

Since both the mark decorator and the method on Metafunc now support id_names, I'd like to keep it that way if the maintainers don't have anything against this.

Regarding an example in the changelog: I've added an example to the documentation. Shouldn't that be enough? At least to me the changelog entry pretty much covers the new feature.

[dongfangtianyu]

To facilitate type checking and autocompletion in the IDE, could you please add the id_names parameter at https://github.com/pytest-dev/pytest/blob/main/src/_pytest/mark/structures.py#L476 ?

[Bastian-Krause]

If you ignore certain parameters, won't the IDs lose their uniqueness?

Yes, a simple example : non unique IDs will be automatically suffixed:

[...]

Hm, I'm not convinced that such special behavior needs to be offered by parametrize() itself. You could use a custom ids function to at least shorten the parameters that make the ID too long.

So I'd like to keep the PR this way for now. You can always extend the functionality later in a separate PR.

To facilitate type checking and autocompletion in the IDE, could you please add the id_names parameter at https://github.com/pytest-dev/pytest/blob/main/src/_pytest/mark/structures.py#L476 ?

Sure! (edit: done)

[The-Compiler]

The CI checks were green before rebasing and seem to happen on the main branch as well.

• Pytest's self tests are failing with pygments v2.19.0 #13112
• Fix selftests with Pygments >= 2.19.0 #13113

Once the fix is in and this PR gets rebased, hopefully things are all green again!

[Bastian-Krause]

I am not sure whether @pytext.fixture() should also support id_names, so tests using parametrized fixtures can also make use of the feature.