Documentation · Concepts

A texture that is a program

A screen that animates is normally a stack of baked frames: seven kinds of console in three colours at eight frames each is 168 images, a hundred and sixty-eight megabytes, and it still steps — because eight frames is all there is. A program is one texture however long it runs, and it does not step. What it costs instead is arithmetic, on the CPU, on your thread, and that cost is two numbers you set.

The first one

A TextureProgram is a small program that computes an image. Hand it source, put it on a material, and the map that would have been a file is computed instead: 

var screen = new TextureProgram("""
    uniform float2 uSize;
    uniform float  uTime;

    half4 main(float2 frag) {
        float2 uv = frag / uSize;
        float bar = step(fract(uv.x + uTime * 0.4), 0.5);
        return half4(half3(0.3, 1.0, 0.5) * half(bar), 1.0);
    }
    """)
{
    Size = new PixelSize(192, 192),
    UpdatesPerSecond = 10
};

console.Material.EmissiveProgram = screen;

That is the whole of it. There is no toolchain, no build step and nothing to ship beside the assembly: the language is SkSL, Skia's own shading language, which is already a dependency of this library on every platform it runs on. It is GLSL-shaped, so the code above will look familiar to anyone who has written a fragment shader, and it is deliberately restricted to what every backend can run.

The entry point is half4 main(float2 frag), and frag is in pixels, not in the nought-to-one coordinates you are probably expecting — divide by uSize for those.

What it returns is premultiplied, which is Skia's rule for every runtime shader and not a choice made here. At full opacity — which is nearly every screen, sign and readout — there is no difference to notice. Where there is one, a half-covered red is half4(half3(1, 0, 0) * 0.5, 0.5), and writing the colour unscaled beside a half alpha asks for a red twice as bright as red. The texture the program fills is straight, as Texture.Pixels promises; the conversion happens on the way in.

The source is compiled once, when you construct it, and a program that will not compile never reaches a driver: Error holds the message, the texture stays transparent, and nothing is evaluated. Check it while you are writing one. The same field speaks if an evaluation ever fails while running: the last picture stays, the program stops, and nothing reaches your frame.

Where it fits, and what the renderers see

Where a texture program sits in a frame On the UI thread a TextureProgram is evaluated per pixel and writes into its own Texture in place, bumping that texture's revision. Below the thread boundary the material's map is an ordinary texture, which the renderer's cache re-uploads when the revision moves — none of the four backends knows that a program exists. UI thread — where the program runs half4 main(float2 frag) Per pixel, UpdatesPerSecond times a second. Texture Same instance, pixels rewritten, revision bumped. Size × Size arithmetic. About 20 ms at 512², 1.3 ms at 128². LastEvaluation says which. the ordinary handover — nothing here is special Render thread — where it is just a map Material.EmissiveTexture Re-uploaded when the revision moved. Metal · Vulkan · OpenGL · Skia None of them knows a program exists. So a program works on every backend, identically, from the day it is written.

A program is a way of filling a texture, not a new kind of material. That is why it needed no change to any of the four renderers, and why there is nothing to check before using one.

Four maps can be computed, and each one keeps the meaning it already had:

BaseColorProgram The albedo map. sRGB-encoded — the renderers linearise it, so write the colour you want to see, not its square.
EmissiveProgram The emissive map, sRGB-encoded and multiplied by EmissiveColor — which must be non-black or the map is multiplied away. Screens, signs and readouts are nearly always this one.
MetallicRoughnessProgram The glTF packing: green is roughness, blue is metallic, red and alpha unused. Linear data, not gamma-corrected. Rust, wear, wet patches, frost.
NormalProgram A tangent-space normal map, linear, +Z out of the surface — so half4(n * 0.5 + 0.5, 1). Needs tangents on the mesh. This is what makes water.

Assigning a program points the corresponding texture at the program's own Texture; clearing it takes the map away again, and changing the program's Size moves the map to the new texture. From there everything downstream — the caches, the uploads, the shaders — treats it as the ordinary texture it is. The texture can also be put on any other map or on a SpriteNode directly, and the view still runs the program; only a resize is not followed there, so assign it again after one.

What a program is given

Four things, and nothing else. Declare what you use; anything you do not declare costs nothing.

uniform float2 uSize; The image's size in pixels. Filled in for you. frag / uSize is the nought-to-one coordinate.
uniform float uTime; The scene's clock in seconds, filled in for you. An absolute reading rather than a step, so a program's phase is a function of the time. A program with no uTime in it is evaluated once and then only when something is set on it, because nothing about it can have changed.
Set(name, …) Your own constants, in the shapes listed below. This is how one piece of source becomes twenty different screens.
SetTexture(name, texture) A map to sample, declared uniform shader name; and read with name.eval(p). The coordinates handed to eval are pixels in this program's image, so scrolling or tiling it is arithmetic on them.

What is not there is the point of the design. A program computes an image, so it never sees the surface's normal, the direction you are looking from, the lights, or the scene behind the surface. Water that ripples its own colour is one of these; water that bends what is behind it is not. Nothing a program does reaches the lighting — it produces a map, and the material does what it has always done with that map.

The constants

One overload per shape SkSL can declare. Each is written into the program's uniform block at the call, and that write is the check.

Set(name, float) uniform float. Write 2f: a bare 2 is an int.
Set(name, int) uniform int.
Set(name, bool) uniform int, as one or zero — SkSL has no bool uniform.
Set(name, Vector2) uniform float2.
Set(name, Vector3) uniform float3.
Set(name, Vector4) uniform float4.
Set(name, Matrix3x2) uniform float3x3, laid out so that m * float3(p, 1) in the program is Vector2.Transform(p, m) in C#.
Set(name, Matrix4x4) uniform float4x4, so that m * float4(p, 1) is Vector4.Transform(p, m).
Set(name, float[]) uniform float name[N]. Exactly N values; the array is copied.
Set(name, Vector4[]) uniform float4 name[N]. Exactly N vectors.
Set(name, SKColorF) uniform float4 — or a float3, which takes the colour without its alpha.
Unset(name) Back to zero until it is set again.

The shape is checked at the call. A Vector4 set on a uniform float throws an ArgumentException that names both sides — uniform 'clock' is declared float; Set(string, Vector4) was called — on your thread, at the line that is wrong, and never from inside a frame. A name the program does not have is not an error: it is noted once in Diagnostics and ignored, because one piece of source serves many screens, and a constant a program has stopped reading is a drift to report rather than a frame to lose. And set every constant you declare: a slot never written holds whatever memory it was given, not zero.

The two dials, which are the whole cost

Evaluation is per pixel, on the CPU, on the UI thread. There is nothing hidden to find: the cost is the number of pixels times the number of evaluations a second, and both are properties on the object. On a quiet machine one evaluation is about 20 ms at 512², and 1.3 ms at 128². 

screen.Size = new PixelSize(128, 128);   // detail — halve it and you quarter the work
screen.UpdatesPerSecond = 8;                // rate — 30 by default, 0 means every frame

Size is detail, not size in the scene: how large the surface appears is the mesh's business. A console across a room does not need more than 128, and it reads the same at eight updates a second as at sixty. Twenty screens at 128² and 8 Hz is about two milliseconds of work a second; twenty at 512² and 30 Hz is not affordable at all, and the difference between those two sentences is entirely in these two lines.

LastEvaluation reports what one evaluation cost and Evaluations how many there have been. Multiply them and that is what the program has spent.

Two things are handled for you. A program is given its own phase of its update period, so twenty screens created together at thirty a second do not all come due on the same frame and cost twenty evaluations in one hitch. And a program whose source has no clock in it is evaluated once, however high the rate is set.

Randomness that survives the crossing

Three functions are available to every program without being declared: 

float ava3dHash(float x);      // 0..1 from a number
float ava3dHash2(float2 p);    // 0..1 from a point
float ava3dNoise(float2 p);    // smooth value noise

Use them rather than the hash out of every shader tutorial. fract(sin(x) * 43758.5453) is the one thing measured across two backends here that did not survive the crossing: the same picture came back with a mean channel difference of 0.057 against a bar of 0.002, because multiplying by forty-three thousand and keeping the fraction turns one unit in the last place of sin into a different random number. The permutation behind ava3dHash keeps every intermediate an exact integer in a float; the same picture then agreed to 0.000028.

These are prepended to your source, and the line numbers in Error are corrected for them, so the number in the message is the line in the code you wrote.

Changing one, and putting it away

Setting a constant or a texture marks the program dirty, and it is evaluated on the next tick whatever its rate says — a knob turned by a user does not wait a third of a second for its picture. One piece of source and one constant per surface is how a bank of screens is built: 

foreach (var (console, phase) in consoles)
{
    var program = new TextureProgram(Source) { Size = new PixelSize(128, 128) };
    program.Set("uConst", new Vector4(phase, 0, 0, 0));
    console.Material.EmissiveProgram = program;
}

Each surface needs its own instance, because a program owns one texture. What differs between them is a handful of numbers.

A program is disposable, and the reason is that a compiled effect is a native object. A game that builds a screen when a room is entered and drops it when the room is left would otherwise leave one behind each time, released only when a finaliser eventually ran. Disposing a program that is still on a material leaves the map as it was — the last picture it drew — and stops it changing.

One neighbouring fact, because it is the thing people meet next: a material is not watched. Assigning or clearing a program is noticed, and so is anything a program does; changing something else on a material that is already on screen is not, and Material.Refresh() is how you say so. The demo's baked screen, which steps its atlas by moving UvOffset, needs exactly that call.

Four things this turns out to be

What it is not is a way into the lighting. A fragment program that participates in shading — with the interpolated normal, the view vector and the tangent frame — is a different feature and is not in this release. If you find yourself wanting the scene behind the surface, that is the boundary you have reached.

Run it

Three programs against a baked eight-frame atlas of the same screen. Watch the right-hand one step.

AVA3D_SCENE="Programmable screens" dotnet run --project samples/Ava3D.Demo.Desktop

The same scene with the probe, which prints what each frame cost:

AVA3D_SCENE="Programmable screens" AVA3D_PROBE=8 dotnet run --project samples/Ava3D.Demo.Desktop