Speeding up Tempest by Pipeline Caching

Published March 2023

Vulkan pipeline creation can get expensive, especially once a project has a lot of pipelines.

Tempest uses VkPipelineCache to make startup a little smoother. The cache is saved to disk and loaded again next time the engine starts.

This is mainly useful on desktop/editor builds. On mobile it’s disabled.

# How it works

Load the cache file (if it exists)
Validate it against the current GPU + driver
Create a VkPipelineCache (from file data or empty)
Pass it into vkCreateGraphicsPipelines
On shutdown, save the updated cache back to disk

Pipeline caching is disabled on mobile. Cache behavior varies too much across devices and updates to be worth dealing with.

# Validation and regeneration

Pipeline caches are not portable. A driver update, a new GPU, or even moving to a different machine can invalidate them.

Tempest checks the cache header and only accepts it if these match:

vendorID
deviceID
pipelineCacheUUID

If the cache is invalid, it’s deleted and rebuilt on the next run. The cache can also grow over time, so it’s fine to regenerate it when needed.

# Saving the cache

void
TempestVulkanDevice::StorePipelineCache()
{
    byte_buffer Buffer;
    vkGetPipelineCacheData(m_device, m_pipelineCache, &Buffer.Size, NULL);
    Buffer.AllocateZeroes(Buffer.Size);
    vkGetPipelineCacheData(m_device, m_pipelineCache, &Buffer.Size, Buffer.Data);

    stack_string_1024 Path("Internal/PipelineCache.bin");
    EngineDeleteFile(Path.ToPathString().GetFullPath().String);
    EngineWriteFile(Path.ToPathString().GetFullPath().String, Buffer.Data, Buffer.Size);
}

# Creating the cache

On desktop/editor builds, Tempest loads the cache from disk and validates it. If it matches the current device properties, it’s used during pipeline creation.

If it doesn’t match, the cache file is deleted so it regenerates cleanly.

On mobile builds, caching is disabled entirely.

void
TempestVulkanDevice::CreatePipelineCache()
{
    #ifdef ENGINE_PLATFORM_MOBILE
    // Pipeline caching disabled on mobile targets
    VkPipelineCacheCreateInfo pipelineCache{};
    pipelineCache.sType = VK_STRUCTURE_TYPE_PIPELINE_CACHE_CREATE_INFO;

    VkResult res = vkCreatePipelineCache(m_device, &pipelineCache, NULL, &m_pipelineCache);
    assert(res == VK_SUCCESS);
    return;
    #endif

    b32 CacheIsValid = false;
    byte_buffer Buffer = {};

    stack_string_1024 Path("Internal/PipelineCache.bin");
    engine_file File = {};

    if (EngineFileExists(Path.ToPathString().GetFullPath().String))
    {
        EngineReadFile(Path.ToPathString().GetFullPath().String, &File);
        Buffer = File.CopyToByteBuffer();
        EngineCloseFile(File);

        if (Buffer.Data && Buffer.Size >= sizeof(VkPipelineCacheHeaderVersionOne))
        {
            VkPipelineCacheHeaderVersionOne* Header =
                (VkPipelineCacheHeaderVersionOne*)Buffer.Data;

            if (Header->deviceID == m_deviceProperties.deviceID &&
                Header->vendorID == m_deviceProperties.vendorID &&
                memcmp(Header->pipelineCacheUUID,
                       m_deviceProperties.pipelineCacheUUID,
                       VK_UUID_SIZE) == 0)
            {
                CacheIsValid = true;
            }
            else
            {
                CacheIsValid = false;

                // deleting the invalid cache file so it regenerates next run
                EngineDeleteFile(Path.ToPathString().GetFullPath().String);
            }
        }
    }

    VkPipelineCacheCreateInfo pipelineCache{};
    pipelineCache.sType = VK_STRUCTURE_TYPE_PIPELINE_CACHE_CREATE_INFO;
    pipelineCache.pNext = NULL;
    pipelineCache.initialDataSize = CacheIsValid ? Buffer.Size : 0;
    pipelineCache.pInitialData = CacheIsValid ? Buffer.Data : NULL;
    pipelineCache.flags = 0;

    VkResult res = vkCreatePipelineCache(m_device, &pipelineCache, NULL, &m_pipelineCache);
    assert(res == VK_SUCCESS);

    Buffer.Destroy();
}

# Notes

Pipeline caches only help after the first run, since they need to be generated.
Caches are not permanent. Driver updates can invalidate them.
Tempest regenerates the cache automatically by deleting invalid files.
Pipeline caching is disabled on mobile.