Distribution Installer

There are quite a number of occasions where it would be useful to download various versions SDK or distributions from a variety of sources and then install them locally without having to affect the environment of a user. The Gradle Wrapper is already a good example of this. Obviously it would be good if one could also utilise other solutions that manage distributions and SDKs on a per-user basis such as the excellent SDKMAN!.

The AbstractDistributionInstaller abstract class provides the base for plugin developers to add such functionality to their plugins without too much trouble.

Getting started

TestInstaller.groovy
class TestInstaller extends AbstractDistributionInstaller {
@CompileStatic
class MyTestInstaller extends AbstractDistributionInstaller {

    public static final String DISTPATH = 'foo/bar'
    MyTestInstaller(ProjectOperations projectOperations) {
        super('Test Distribution', DISTPATH, ConfigCacheSafeOperations.from(projectOperations)) (1)
    }

    @Override
    URI uriFromVersion(String version) { (2)
        "https://distribution.example/download/testdist-${DISTVER}.zip".toURI() (3)
    }
}
}
1 The installer needs to be provided with a human-readable name and a relative path below the installation for installing this type of distribution.
2 The uriFromVersion method is used to return an appropriate URI where to download the specific version of distribution from. Supported protocols are all those supported by Gradle Wrapper and includes file, http(s) and ftp.
3 Use code appropriate to your specific distribution to calculate the URI.

The download is invoked by calling the getDistributionRoot method.

The above example uses Groovy to implement an installer class, but you can use Java, Kotlin or any other JVM-language that works for writing Gradle plugins.

How it works

When getDistributionRoot is called, it effectively uses the following logic

File location = locateDistributionInCustomLocation(version) (1)

if (location == null && sdkManCandidateName) { (2)
    location = getDistFromSdkMan(version)
}

location ?: getDistFromCache(version) (3)
1 If a custom location is specified, look there first for the specific version
2 If SDKMAN! has been enabled, look if it has an available distribution.
3 Try to get it from cache. If not in cache try to download it.

Marking files executable

Files in some distributed archives are platform-agnostic and it is necessary to mark specific files as executable after unpacking. The addExecPattern method can be used for this purpose.

addExecPattern('**/*.sh') (1)
1 Assuming the TestInstaller from Getting Started, this example will mark all shell files in the distribution as executable once the archive has been unpacked.

Patterns are ANT-style patterns as is common in a number of Gradle APIs.

Search in custom locations

The locateDistributionInCustomLocation method can be used for setting up a search in specific locations.

For example a person implementing a Ceylon language plugin might want to look in the ~/.ceylon folder for an existing installation of a specific version.

This optional implementation is completely left up to the plugin author as it will be very specific to a distribution. The method should return null if nothing was found.

Changing the download and unpack root location

By default, downloaded distributions will be placed in a subfolder below the Gradle user home directory as specified during construction time. It is possible, especially for testing purposes, to use a root folder other than Gradle user home by setting the downloadRoot

Utilising SDKMAN!

SDKMAN! is a very useful local SDK installation and management tool and when specific SDKs or distributions are already supported it makes sense to re-use them in order to save on download time.

All that is required is to provide the SDKMAN! candidate name using the setSdkManCandidateName method.

Utilising SDKMAN!
installer.sdkManCandidateName = 'ceylon' (1)
1 Sets the candidate name for a distribution as it will be known to SDKMAN!. In this example the Ceylon language distribution is used.

Checksum

By default the installer will not check any values, but calling setChecksum will force the installer to perform a check after downloading and before unpacking. It is possible to invoke a behavioural change by overriding verification.

Only SHA-256 checksums are supported. If you need something else you will need to override verification and provide your own checksum test.

Advanced: Override unpacking

By default, AbstractDistributionInstaller already knows how to unpack ZIPs and TARs of a variety of compressions. If something else is required, then the unpack method can be overridden.

There is also support for XZ, MSI (Windows-only), and DMG (Mac only) formats. This is achieved by placing the appropriate artifact org.ysb33r.gradle:grolifant5-unpacker-<EXT> on the runtime classpath.

XZ

build.gradle
dependencies {
  runtimeOnly 'org.ysb33r.gradle:grolifant5-unpacker-xz:5.2.5'
}

OR with version catalog

gradle/libs.versions.toml
[libraries]
grolifantUnpackerXz = { module = "org.ysb33r.gradle:grolifant5-unpacker-xz", version.ref = "grolifant" }
build.gradle
dependencies {
  runtimeOnly libs.grolifantUnpackerXz
}

MSI

build.gradle
dependencies {
  runtimeOnly 'org.ysb33r.gradle:grolifant5-unpacker-msi:5.2.5'
}

OR with version catalog

gradle/libs.versions.toml
[libraries]
grolifantUnpackerMsi = { module = "org.ysb33r.gradle:grolifant5-unpacker-msi", version.ref = "grolifant" }
build.gradle
dependencies {
  runtimeOnly libs.grolifantUnpackerMsi
}

This unpacker relies on LessMsi. The tool is automatically installed and the version used is 2.2.0. It is possible to override this version by setting org.ysb33r.grolifant5.lessmsi.version.

It is also possible to control the environment for lessmsi. Override

@Override
@Nullable
protected GrolifantUnpacker.Parameters unpackParametersForExtension(String extension) {
  if(extension.toLowerCase(Locale.US) == 'msi') {
    final params = new UnpackerParameters(configCacheSafeOperations)
    params.environment( foo: 'bar')
    params
  } else {
    super.unpackParametersForExtension(extension)
  }
}

DMG

In a similar fashion DMGs can be unpacked on Mac OSX platforms. It used hdiutil under the hood.

build.gradle
dependencies {
  runtimeOnly 'org.ysb33r.gradle:grolifant5-unpacker-dmg:5.2.5'
}

OR with version catalog

gradle/libs.versions.toml
[libraries]
grolifantUnpackerDmg = { module = "org.ysb33r.gradle:grolifant5-unpacker-dmg", version.ref = "grolifant" }
build.gradle
dependencies {
  runtimeOnly libs.grolifantUnpackerDmg
}

Installing single files

In some cases, tools are supplied as single executables. terraform and packer are such examples. Use AbstractSingleFileInstaller instead.

class TestInstaller extends AbstractSingleFileInstaller { (1)
    TestInstaller(final ProjectOperations po) {
        super('mytool', 'native-binaries/mytool', po) (2)
    }

    @Override
    protected String getSingleFileName() { (3)
        OperatingSystem.current().windows ? 'mytool.exe' : 'mytool'
    }
}
1 Extend AbstractSingleFileInstaller instead
2 The version is no longer used in the constructor of the super class.
3 Implement a method to obtain the name of the file.

You can access a downloaded single file by version. Simply call getSingleFile(version).

Advanced: Override verification

Verification of a downloaded distribution occurs in two parts:

  • If a checksum is supplied, the downloaded archive is validated against the checksum. The standard implementation will only check SHA-256 checksums.

  • The unpacked distribution is then checked for sanity. In the default implementation this is simply to check that only one directory was unpacked below the distribution directory. The latter is effectively just replicating the Gradle Wrapper behaviour.

Once again it is possible to customise this behaviour if your distribution have different needs. In this case there are two protected methods than can be overridden:

  • verifyDownloadChecksum - Override this method to take care of handling checksums. The method, when called, will be passed the URI where the distribution was downloaded from, the location of the archive on the filesystem and the expected checksum. It is possible to pass null for the latter which means that no checksum is available.

  • getAndVerifyDistributionRoot - This validates the distribution on disk. When called, it is passed the the location where the distribution was unpacked into. The method should return the effective home directory of the distribution.

In the case of getAndVerifyDistributionRoot it can be very confusing sometimes as to what the distDir is and what should be returned. The easiest is to explain this by looking at how Gradle wrappers are stored. For instance for Gradle 7.0 the distDir might be something like ~/.gradle/wrapper/dists/gradle-7.0-bin/2z3tfybitalx2py5dr8rf2mti/ whereas the return directory would be ~/.gradle/wrapper/dists/gradle-7.0-bin/2z3tfybitalx2py5dr8rf2mti/gradle-7.0.

Helper and other protected API methods

  • listDirs provides a listing of directories directly below an unpacked distribution. It can also be used for any directory if the intent is to see which child directories are available.