Testing plugins

Testing your plugin should follow the normal technique of using Gradle’s Testkit, and optionally use GradleTest plugin for more coverage. Due to the multiple runtime variants of grolifant-herd you will probably pick up issues when testing outside of the Gradle version family that you are using during development. For instance, you might be compiling against Gradle 8.x, but you might want to test against Gradle 7.x or Gradle 9.x. GradleTest simplifies this, but you can still do this purely with Testkit if you are willing to put in the work.

GradleTest

plugins {
  id 'org.ysb33r.gradletest' version '5.12.2' (1)
}

gradleTestSets {
  testSets {
    main {
        versions '7.6.6', '8.14.5'
        useCustomManifestPerVersion {
            useGradleApiVersion {
                it.startsWith('7.') ? '7.3.3' : '8.0' (2)
            }
        }
    }
  }
}

1	Use any GradleTest version 5.1.1 or later.
2	This is a rule that sets the Gradle API attribute to a specific family version. For more details see Custom Manifests in the GradleTest documentation.

Testkit only

For Testkit you need to:

Set up a separate resolvable configuration.
Adjust the attributes on the configuration.
Create a task that will create a custom manifest.
Run TestKit with the custom classpath ` .build.gradle

configurations {
  resolvable('customManifest') { (1)
    extendsFrom('runtimeClasspath') (2)
    attributes {
      attribute(GradlePluginApiVersion.GRADLE_PLUGIN_API_VERSION_ATTRIBUTE,objects.named(GradlePluginApiVersion, '7.3.3')) (3)
      attribute(TargetJvmVersion.TARGET_JVM_VERSION_ATTRIBUTE, 17) (4)
    }
  }
}

final customManifestOutputFileProvider = project.layout.buildDirectory.file('pluginUnderTestMetadata/custom-manifest.txt') (5)
tasks.register('customManifest') {

  final classpath = objects.fileCollection()
  final rt = sourceSets.main
  classpath.from(configurations.customManifest) (6)
  classpath.from(rt.classesDirs) (7)
  classpath.from(rt.resourcesDir)

  outputs.file(customManifestOutputFileProvider)
  inputs.files(classpath)

  doLast {
        customManifestOutputFileProvider.get().asFile.text = classpath.files*.absolutePath.join('\n') (8)
  }
}

tasks.named('test', Test) {
  jvmArgumentProviders.add( { -> ["-DMANIFEST_FILE=${customManifestOutputFileProvider.get().absolutePath}"]*.toString() } as CommandLineArgumentProvider) (9)
  dependsOn 'customManifest' (10)
}

1	Create a resolvable configuration
2	It needs to have access to the same dependencies that the `runtimeClasspath` has.
3	Set the Gradle API attribute to something that matches the Gradle version family.
4	Optionally set the Java runtime attribute if the JDK version that runs the test mismatches the Grolifant variant you need.
5	Location of the custom manifest file
6	Custom manifest needs the files resolved in the custom configuration.
7	Custom manifest also needs the main source set’s outputs.
8	Create the actual custom manifest
9	Pass the location of the manifest file to the test suite.
10	Ensure the manifest is created before the tests run.

And in addition, you’ll need to add seem scaffolding to your test.

MySpockTest.groovy

class MySpockTest extends Specification {
    public static final File MANIFEST_FILE = new File(System.getProperty('MANIFEST_FILE')) (1)

   void 'my test'() {
     setup:
     final classpath = MANIFEST_FILE.readLines().collect { new File(it) } (2)
     final gradleRunner = GradleRunner.create()
        .withGradleVersion('7.3.3') (3)
        .withPluginClasspath(classpath) (4)

     // .....
   }
}

1	Get the location of the manifest file.
2	Turn the file contents into a list of files.
3	Set the custom Gradle version to test.
4	Use the custom classpath.

As one can see there is quite a bit of setup and therefore the recommendation is to perform unit and integration testing within the specific Gradle version family and then to defer wider testing to GradleTest.