Every Verso feature is an extension, and a Perl kernel proves it

A month ago we published four rules for Verso. The second one was that every built-in feature has to be implemented on the same public interfaces any third party gets, with no private APIs for our own code.

Rules like that are easy to write and easy to quietly stop keeping. What I like about this one is that you can check it. Point at a built-in, ask which public interface it implements, and see whether the answer is a real name from a package anyone can install.

Here is the whole list, as of today's release.

The last two are supplemental: they get implemented alongside a primary capability rather than standing on their own. One of them, ICellInteractionHandler, has no built-in behind it yet. I am leaving that row in, because an interface with no first-party consumer is an interface nobody on our side has had to live with.

The cheapest proof I could think of

Today's release, 1.0.9, adds a sample called Verso.Sample.Perl. It is a language kernel for Perl, and it works the only way a kernel for Perl reasonably can: it writes the cell to a temporary .pl file and runs the system perl against it.

[VersoExtension]
public sealed class PerlKernel : ILanguageKernel
{
    public string ExtensionId => "com.verso.sample.perl";
    public string Name => "Perl Kernel";
    public string Version => "1.0.0";
    public string LanguageId => "perl";
    public string DisplayName => "Perl";
    public IReadOnlyList<string> FileExtensions => new[] { ".pl", ".pm" };

    public async Task<IReadOnlyList<CellOutput>> ExecuteAsync(
        string code, IExecutionContext context)
    {
        var tempFile = Path.GetTempFileName() + ".pl";
        await File.WriteAllTextAsync(tempFile, code, context.CancellationToken);

        using var process = Process.Start(new ProcessStartInfo("perl", tempFile)
        {
            RedirectStandardOutput = true,
            RedirectStandardError = true,
            UseShellExecute = false,
            CreateNoWindow = true
        })!;

        await using var registration = context.CancellationToken.Register(
            () => process.Kill(entireProcessTree: true));

        var stdout = await process.StandardOutput.ReadToEndAsync();
        await process.WaitForExitAsync(context.CancellationToken);

        return new[] { new CellOutput("text/plain", stdout) };
    }
}

That is abridged. The real file also captures standard error as an error output, deletes the temporary file, and reports cancellation. The point is what is not in it. No registration call into the engine. No switch statement somewhere that knows Perl exists. No entry in a list of blessed languages. The host finds the class because it carries [VersoExtension] and implements ILanguageKernel, and from that moment Perl is a choice in the cell language picker.

The rest of the interface is the more interesting part, because ILanguageKernel asks for more than execution. GetDiagnosticsAsync runs perl -c on the same temporary file and turns the compiler's complaint into a Diagnostic on the right line. GetCompletionsAsync returns a list of snippets. GetHoverInfoAsync reports the interpreter version detected during InitializeAsync. So you get squiggles, completions and hover in a Perl cell, out of a sample of about two hundred and thirty lines that references exactly one package.

It is a sample, not a supported kernel. Its job is to be the thing we would have to delete if the claim stopped being true.

What the rule costs

The bill arrives in the loader. Because a built-in and a stranger's package implement the same interfaces, the engine has to be able to load both, and they don't want the same treatment.

Built-in extensions sit next to the engine and load with no isolation. They were compiled against the same Verso.Abstractions the host is running, so isolating them would buy nothing. A third-party extension gets its own load context, which can be unloaded, and which resolves the extension's own dependencies out of its own folder rather than fighting with the host's.

That isolation creates the problem it is supposed to solve. An extension compiled against its own copy of Verso.Abstractions would produce an ILanguageKernel that is not the host's ILanguageKernel, and the cast would fail for reasons nobody enjoys debugging. So the load context intercepts that one assembly name and hands back the host's own instance, whatever version the extension was built against.

Which then needs a rule about versions. This release adds a compatibility check that reads the referenced version out of the assembly and compares it with the host's. A different major version is refused outright. A higher minor version is refused with a message asking for a newer host, because the extension may be using interface members this host has never heard of. Patch versions are not compared at all. An incompatible extension is skipped during discovery, or throws when you load it by path and ask for it directly.

What it is like to write one

None of that is visible when you write one. The template scaffolds a project that references Verso.Abstractions and nothing else:

dotnet new install Verso.Templates
dotnet new verso-extension -n MyExtension \
    --extensionId com.mycompany.myext \
    --author "Your Name"

And a cell loads your build output straight from disk, with the path resolved relative to the notebook:

#!extension ./MyExtension/bin/Debug/net8.0/MyExtension.dll

Build, re-run the cell, and the new layout or theme or kernel is registered in the running session. A NuGet package id works in the same command and asks for consent first, since that one reaches the network.

I don't think this rule makes Verso better in a way you can feel in the first ten minutes. It makes it harder for us to paint ourselves into a corner later. A built-in that quietly reaches for an internal API is a feature nobody outside can reproduce, and it is also a signal that the interface it should have used is missing something. I would much rather find that out from a small Perl sample than from your bug report.