Using an interface in dynamic data causes a de-/serialization exception

[StefanOssendorf]

Description

Upgrading MSTest.TestAdapter from version 2.2.3 to 2.2.5 causes this error.
To test a extension method I am providing some dynamic data as IEnumerable which now provides an exception.

Steps to reproduce

// Test code which produces the error
public static IEnumerable<Object[]> AddTestCases {
    get {
        var sut = new Collection<String>();
        var expected = new Collection<String>();
        yield return new Object[] { sut, Enumerable.Empty<String>(), expected };

        sut = new Collection<String>() { "1" };
        expected = new Collection<String>() { "1" };
        yield return new Object[] { sut, Enumerable.Empty<String>(), expected };

        sut = new Collection<String>();
        expected = new Collection<String>() { "1" };
        yield return new Object[] { sut, new[] { "1" }, expected };

        sut = new Collection<String>() { "1", "a", "b" };
        expected = new Collection<String>() { "1", "a", "b", "z", "j" };
        yield return new Object[] { sut, new[] { "z", "j" }, expected };
    }
}

[DataTestMethod]
[DynamicData(nameof(AddTestCases))]
public void Add_ShouldAddTheExpectedValues(Collection<String> systemUnderTest, IEnumerable<String> itemsToAdd, Collection<String> expected) {
    // The acutal tested method is irrelevant. Executing this empty tests provokes the error
}

Expected behavior

4 test cases are executed

Actual behavior

Only two test cases are executed and then the following exception is logged from dotnet test.

Failed Add_ShouldAddTheExpectedValues (System.Collections.ObjectModel.Collection`1[System.String],System.Linq.EmptyPartition`1[System.String],System.Collections.ObjectModel.Collection`1[System.String])
  Error Message:
   Value cannot be null. (Parameter 'con')
  Stack Trace:
     at System.Reflection.Emit.DynamicILGenerator.Emit(OpCode opcode, ConstructorInfo con)
   at System.Runtime.Serialization.CodeGenerator.New(ConstructorInfo constructorInfo)
   at System.Runtime.Serialization.Json.JsonFormatReaderGenerator.CriticalHelper.ReadCollection(CollectionDataContract collectionContract)
   at System.Runtime.Serialization.Json.JsonFormatReaderGenerator.CriticalHelper.GenerateCollectionReader(CollectionDataContract collectionContract)
   at System.Runtime.Serialization.Json.JsonCollectionDataContract.get_JsonFormatReaderDelegate()
   at System.Runtime.Serialization.Json.JsonCollectionDataContract.ReadJsonValueCore(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
   at System.Runtime.Serialization.Json.JsonDataContract.ReadJsonValue(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
   at System.Runtime.Serialization.Json.XmlObjectSerializerReadContextComplexJson.ReadDataContractValue(DataContract dataContract, XmlReaderDelegator reader)
   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
   at System.Runtime.Serialization.XmlObjectSerializerReadContextComplex.InternalDeserialize(XmlReaderDelegator xmlReader, Type declaredType, DataContract dataContract, String name, String ns)
   at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName)
   at System.Runtime.Serialization.XmlObjectSerializer.InternalReadObject(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
   at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.ReadObject(XmlDictionaryReader reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.ReadObject(Stream stream)

  Failed Add_ShouldAddTheExpectedValues (System.Collections.ObjectModel.Collection`1[System.String],System.Linq.EmptyPartition`1[System.String],System.Collections.ObjectModel.Collection`1[System.String])
  Error Message:
   Value cannot be null. (Parameter 'con')
  Stack Trace:
     at System.Reflection.Emit.DynamicILGenerator.Emit(OpCode opcode, ConstructorInfo con)
   at System.Runtime.Serialization.CodeGenerator.New(ConstructorInfo constructorInfo)
   at System.Runtime.Serialization.Json.JsonFormatReaderGenerator.CriticalHelper.ReadCollection(CollectionDataContract collectionContract)
   at System.Runtime.Serialization.Json.JsonFormatReaderGenerator.CriticalHelper.GenerateCollectionReader(CollectionDataContract collectionContract)
   at System.Runtime.Serialization.Json.JsonCollectionDataContract.get_JsonFormatReaderDelegate()
   at System.Runtime.Serialization.Json.JsonCollectionDataContract.ReadJsonValueCore(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
   at System.Runtime.Serialization.Json.JsonDataContract.ReadJsonValue(XmlReaderDelegator jsonReader, XmlObjectSerializerReadContextComplexJson context)
   at System.Runtime.Serialization.Json.XmlObjectSerializerReadContextComplexJson.ReadDataContractValue(DataContract dataContract, XmlReaderDelegator reader)
   at System.Runtime.Serialization.XmlObjectSerializerReadContext.InternalDeserialize(XmlReaderDelegator reader, String name, String ns, Type declaredType, DataContract& dataContract)
   at System.Runtime.Serialization.XmlObjectSerializerReadContextComplex.InternalDeserialize(XmlReaderDelegator xmlReader, Type declaredType, DataContract dataContract, String name, String ns)
   at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName)
   at System.Runtime.Serialization.XmlObjectSerializer.InternalReadObject(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
   at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.ReadObject(XmlDictionaryReader reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializerImpl.ReadObject(Stream stream)

Environment

Operating system: Windows 10 latest patches
Build version of vstest.console: Microsoft (R) Test Execution Command Line Tool Version 16.10.0
Package version of MSTest framework and adapter: Both 2.2.5 but only updating the Adapater to 2.2.5 causes this issues
Other installed packages and their versions on the test project:

• FluentAssertions 5.10.3
• Microsoft.CodeAnalysis.NetAnalyzers 5.0.3
• Microsoft.NET.Test.Sdk 16.10.0
• coverlet.collector 3.0.3

Note

A workaround is replacing Enumerable.Empty<String>() with Array.Empty<String>().

AB#1408170

[Haplois]

This is by-design. We'll work on make this exception clearer.

All data should be serializable and deserializable. We're working on an opt-out of this behavior, #905. Until that's done you can downgrade to 2.2.3.

Also, I'll look on providing an exception to that rule for IEnumerable interface.

[StefanOssendorf]

Thank you for your quick reply.
Okay I'll keep that in mind for new tests. Thanks!

[Evangelink]

Closing this issue as it will be handled as part of the more global ticket #1462