xunit-maui: can't get any tests to execute.

[ksoftllc]

Trying out your Android runner to automate testing on our MAUI project. I set up a new MAUI project, modified MauiProgram exactly as specified, deleted App.xaml and App.xaml.cs. When I try to run a test, nothing happens. All the tests show as skipped. Test log shows the following:

Building Test Projects
Starting test discovery for requested test run
========== Starting test discovery ==========
You must install or update .NET to run this application.
App: C:\fleet-hd-mobile\src\AndroidTestRunner\bin\Debug\net8.0-android\testhost.exe
Architecture: x64
Framework: 'Microsoft.Android', (x64)
.NET location: C:\Program Files\dotnet\
No frameworks were found.
Learn more:
https://aka.ms/dotnet/app-launch-failed
To install missing framework, download:
https://aka.ms/dotnet-core-applaunch?framework=Microsoft.Android&arch=x64&rid=win-x64&os=win10
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Testhost process for source(s) 'C:\fleet-hd-mobile\src\AndroidTestRunner\bin\Debug\net8.0-android\AndroidTestRunner.dll' exited with error: You must install or update .NET to run this application.
App: C:\fleet-hd-mobile\src\AndroidTestRunner\bin\Debug\net8.0-android\testhost.exe
Architecture: x64
Framework: 'Microsoft.Android', (x64)
.NET location: C:\Program Files\dotnet\
No frameworks were found.
Learn more:
https://aka.ms/dotnet/app-launch-failed
To install missing framework, download:
https://aka.ms/dotnet-core-applaunch?framework=Microsoft.Android&arch=x64&rid=win-x64&os=win10
. Please check the diagnostic logs for more information.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.ThrowOnTestHostExited(IEnumerable`1 sources, Boolean testHostExited) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Client/ProxyOperationManager.cs:line 520
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Client/ProxyOperationManager.cs:line 294
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Client/ProxyDiscoveryManager.cs:line 149
========== Test discovery aborted: 0 Tests found in 29.5 ms ==========
========== Starting test run ==========
========== Test run aborted: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========
Test project AndroidTestRunner does not reference any .NET NuGet adapter. Test discovery or execution might not work for this project.
It's recommended to reference NuGet test adapters in each test project in the solution.

Not sure how to resolve this. Test runner project builds, but all tests are skipped.

Any suggestions would be greatly appreciated.

[ksoftllc]

Here is an update.

I added the AndroidTestRunner modeled after your Sample. I gave it a reference to the mobile app's DLL. When I run, I don't see a test runner as in your example, I see screens from the FleetHDMobile app.

AndroidTestRunner MauiProgram.cs is exactly like the sample. However, if I put a break in it, it never gets hit. It does hit a breakpoint in MauiProgram.cs from the FleetHDMobile app. Not sure how to get it to prefer AndroidTestRunner's MauiProgram.

[aritchie]

This isn't really a project I support any longer. It isn't a test runner, so it is very hard to automate with. You basically have to run as an app as shown in the project sample.

[ksoftllc]

FYI - I was able to get this to work.

[munkii]

Would you mind explaining how you did get it to work

[ksoftllc]

Would you mind explaining how you did get it to work

Here are the key steps that worked for me. I imagine there are other ways to accomplish this, but here is what I did:

• Create an AndroidTestRunner MAUI project within the same solution as your MAUI app you would like to test.
• Create a Test configuration for the app to test
  • Open Build -> Configuration Manager
  • Under your app, add a new Configuration called "Test" and base it on Debug so that you can use the debugger on your unit tests

• You will create a new compile flag called ANDROID_TEST_RUNNER for the Test configuration only. On your app to test, right click the .csproj file in the solution explorer and choose Properties. In the properties, choose Build -> General and find the Test configuration where you can add the DEBUG and ANDROID_TEST_RUNNER flags:

• You will want your test runner to be its own application, so you must leave out the [Application] attribute in your app's MainApplication.cs file by using the new flag:

#if (ANDROID_TEST_RUNNER == false)
[Application]
#endif

• For your AndroidTestRunner, delete the following files:
  • App.xaml & App.xaml.cs
  • AppShell.xaml & AppShell.xaml.cs
  • all of the platform folders except Android
• Add the Shiny.Xunit.Runners.Maui NuGet package
• Replace the entire contents of MauiProgram.cs in your AndroidTestRunner project with the following:

using Xunit.Runners.Maui;

namespace AndroidTestRunner;

public static class MauiProgram
{
    public static MauiApp CreateMauiApp() => MauiApp
       .CreateBuilder()
       .ConfigureTests(new TestOptions
       {
           Assemblies =
           {
                typeof(MauiProgram).Assembly
           }
       })
       .UseVisualRunner()
       .Build();
}

• You will also need to add all of the packages used by your app to test to the AndroidTestRunner project.
• Add a UnitTests folder to AndroidTestRunner project
• You can start with this SampleTests.cs from the Shinyorg repo:

using Xunit;
using Xunit.Abstractions;

namespace UnitTests;

public class SampleTests
{
    readonly ITestOutputHelper _output;

    public SampleTests(ITestOutputHelper output)
    {
        _output = output;
    }

    [Fact]
    public void SuccessfulTest()
    {
        Assert.True(true);
    }

    [Fact(Skip = "This test is skipped.")]
    public void SkippedTest()
    {
    }

    [Fact]
    public void FailingTest()
    {
        throw new Exception("This is meant to fail.");
    }

    [Theory]
    [InlineData(1)]
    [InlineData(2)]
    [InlineData(3)]
    public void ParameterizedTest(int number)
    {
        Assert.NotEqual(0, number);
    }

    [Fact]
    public void OutputTest()
    {
        _output.WriteLine("This is test output.");
    }

    [Fact]
    public void FailingOutputTest()
    {
        _output.WriteLine("This is test output.");
        throw new Exception("This is meant to fail.");
    }
}

• Return to the Configuration Manager by going to Build -> Configuration Manager. Set the app to test to use the Test configuration and the AndroidTestRunner to use the Debug configuration.

• Now build the app to test by right clicking on it in the solution explorer and choosing Build.
• There should now be a Test folder under the app to test's bin folder.
• You should now be able to add a shared project link to your app to test. In the solution explorer, right click on Dependencies under AndroidTestRunner and choose Add Shared Project Reference. Click the Browse button and find the AppToTest.dll file under the bin\Test\net8.0-android folder of your app to test. Click Add button.
• At this point you should be able to run the AndroidTestRunner project on an Android device or simulator and execute the tests.