Shader Service

Overview

The ShaderService provides runtime GLSL-to-SPIR-V compilation with caching and hot-reload capabilities for the Waxed compositor. It allows plugins to register shader files that are automatically compiled, cached, and monitored for changes.

Key Features:

• Runtime GLSL compilation via glslangValidator
• SPIR-V disk caching by source hash
• inotify-based hot reload
• RAII-managed Vulkan shader modules using vulkan_raii.hpp
• Reference counting for shared shaders
• Thread-safe compilation queue

ShaderStage Enum

enum class ShaderStage : uint32_t {
    Vertex = 0,
    Fragment = 1,
    Compute = 2,
};

Each stage maps to a Vulkan shader stage flag and a glslang argument string:

ShaderStage	Vulkan Flag	glslang Arg
Vertex	eVertex	vert
Fragment	eFragment	frag
Compute	eCompute	comp

Conversion Functions

constexpr auto shader_stage_to_vulkan(ShaderStage stage) -> vk::ShaderStageFlagBits;
constexpr auto shader_stage_to_glsl_arg(ShaderStage stage) -> const char*;

ShaderHandle

Shaders are identified by a uint64_t handle value:

using ShaderHandle = uint64_t;
constexpr ShaderHandle INVALID_SHADER_HANDLE = 0;

Handles are monotonically increasing values starting from 1, generated by an atomic counter.

Configuration

struct Config {
    std::string cache_dir;              // Directory for SPIR-V cache files
    bool enable_hot_reload = true;      // Enable inotify-based hot reload
    size_t compilation_queue_size = 64; // Max pending compilation requests
};

Example Configuration:

ShaderService::Config config;
config.cache_dir = "/path/to/cache/shaders";  // Directory for SPIR-V cache
config.enable_hot_reload = true;

ShaderService shader_service(config);

API Reference

Initialization

auto init_vulkan(vk::raii::Device& device) -> Result<void>;

Initializes the ShaderService with a Vulkan device. Must be called before watch_shader(). Starts the hot-reload watcher thread if enabled.

watch_shader()

auto watch_shader(const std::string& glsl_path,
                 ShaderStage stage,
                 ShaderCallback callback,
                 const std::string& plugin_name) -> std::optional<ShaderHandle>;

Registers a GLSL shader for compilation and monitoring.

Parameters:

• glsl_path - Absolute path to the .glsl source file
• stage - Shader stage (Vertex, Fragment, or Compute)
• callback - Function to call when shader is hot-reloaded
• plugin_name - Name of the plugin registering the shader (for logging)

Returns: A ShaderHandle on success, std::nullopt on failure.

Callback Signature:

using ShaderCallback = std::function<void(vk::raii::ShaderModule* new_module, const std::string& glsl_path)>;

Important: The vk::raii::ShaderModule* pointer is only valid during the callback. Do not store it. Extract the raw handle if needed:

auto callback = [](vk::raii::ShaderModule* new_module, const std::string& path) {
    VkShaderModule raw_handle = **new_module;  // Extract for storage
    // Update pipeline cache or descriptor sets
};

get_shader_module()

auto get_shader_module(ShaderHandle handle) const -> std::optional<VkShaderModule>;

Retrieves the raw Vulkan shader module handle for pipeline creation.

Returns: The raw VkShaderModule handle, or std::nullopt if the handle is invalid.

auto module = shader_service.get_shader_module(handle);
if (module) {
    vk::PipelineShaderStageCreateInfo stage_info{};
    stage_info.module = *module;
    // ... configure pipeline
}

unwatch_shader()

auto unwatch_shader(ShaderHandle handle) -> ShaderResult<void>;

Decrements the reference count for a shader. When the count reaches zero:

• The shader module is destroyed (RAII)
• The inotify watch is removed
• The shader entry is erased from the registry

Note: Multiple plugins can watch the same shader (same path + stage). They share a single ShaderHandle with reference counting.

process_compilation_queue()

auto process_compilation_queue() -> void;

Processes pending hot-reload compilation requests. Called by the main render loop to compile changed shaders on the main thread.

cleanup()

auto cleanup() -> void;

Stops the watcher thread, removes all inotify watches, closes the inotify file descriptor, and clears the shader registry. Called automatically by the destructor.

Shader Compilation Pipeline

Cache Lookup Flow

Hot Reload Sequence

watch_shader() Flow with Reference Counting

SPIR-V Caching

Hash Computation

auto compute_source_hash(const std::string& glsl_path) const -> std::optional<std::string>

1. Reads entire GLSL source file
2. Computes std::hash<std::string> on the source content
3. Formats hash as 16-character hex string
4. Returns the hash string for cache path construction

Note: Not cryptographically secure, but sufficient for cache invalidation. Consider SHA-256 for production.

Cache File Location

<cache_dir>/<hash>.spv

Example: /path/to/cache/shaders/a4f2e1b3c8d9f2a1.spv

Cache Invalidation

The cache is implicitly invalidated when the GLSL source changes, because:

1. Modified source → different hash
2. Different hash → different cache path
3. Cache miss triggers recompilation
4. New SPIR-V written to new cache path

Old cache files accumulate and can be cleaned by a separate cache pruning utility.

glslang Compilation

auto compile_glsl(const std::string& glsl_path,
                 ShaderStage stage,
                 std::vector<uint32_t>& spirv) const -> ShaderResult<void>

Compilation Command:

glslangValidator -V -S <stage> "<glsl_path>" -o "/tmp/waxed_glsl_XXXXXX.spv" 2>&1

Process:

1. Creates a temporary file using mkstemps()
2. Executes glslangValidator via std::system()
3. Reads compiled SPIR-V from temporary file
4. Validates SPIR-V magic number (0x07230203)
5. Converts byte buffer to uint32_t vector
6. Cleans up temporary file

Exit Codes:

• 0 - Success
• Non-zero - Compilation failed (returns ShaderErrorCode::CompilationFailed)

inotify-based Hot Reload

Setup

auto setup_inotify() -> ShaderResult<void>

Initializes inotify with IN_NONBLOCK flag:

inotify_fd_ = inotify_init1(IN_NONBLOCK);

Watch Registration

When watch_shader() succeeds, an inotify watch is added:

int wd = inotify_add_watch(inotify_fd_, glsl_path.c_str(), IN_MODIFY);

• Watch descriptor mapped to ShaderHandle in inotify_to_handle_
• Monitors IN_MODIFY events (file content changes)

Watcher Thread

auto watcher_thread_loop() -> void

Runs in a background thread:

1. Blocks on read(inotify_fd_, buffer, sizeof(buffer))
2. Parses inotify_event structures
3. Maps watch descriptor to shader handle
4. Queues compilation request
5. Notifies condition variable

Thread Safety

• shaders_mutex_ - Protects shader registry (shared_mutex for read-heavy)
• queue_mutex_ - Protects compilation queue
• inotify_mutex_ - Protects watch descriptor mapping

Compilation Queue

Structure

struct CompilationRequest {
    ShaderHandle handle;
    std::string glsl_path;
    ShaderStage stage;
};

std::vector<CompilationRequest> compilation_queue_;

Queueing

Triggered by inotify events in the watcher thread:

{
    std::lock_guard lock(queue_mutex_);
    compilation_queue_.push_back(req);
}
queue_cv_.notify_one();

Processing

Called from main render loop:

auto process_compilation_queue() -> void

Drains queue and compiles each shader. Runs on main thread to ensure Vulkan device access is thread-safe.

ShaderCallback Notification

Callback Timing

The callback is invoked after the shader module is successfully replaced in the registry:

// 1. Compile new module
auto new_module = create_shader_module(spirv);

// 2. Atomically replace in registry
it->second.module = std::move(new_module);

// 3. Get pointer for callback
vk::raii::ShaderModule* ptr = it->second.module.get();

// 4. Unlock before callback
lock.unlock();

// 5. Invoke callback (outside lock)
callback(ptr, glsl_path);

Callback Safety

• Pointer is valid only during callback
• Do not store the pointer
• Extract raw handle if persistence needed:

  VkShaderModule raw = **new_module_ptr;

ShaderErrorCode

All shader operations return ShaderResult<T> = std::expected<T, ShaderErrorCode>:

Code	Value	Description
Success	0	Operation succeeded
GlslFileNotFound	1	GLSL file doesn’t exist
GlslFileReadFailed	2	Cannot read GLSL file
CacheDirCreateFailed	3	Cannot create cache directory
CacheWriteFailed	4	Cannot write cache file
CacheReadFailed	5	Cannot read cache file
CompilationFailed	20	glslangValidator failed
GlslangNotFound	21	glslangValidator not in PATH
GlslangExecuteFailed	22	glslangValidator execution error
InvalidShaderStage	23	Invalid shader stage enum
ShaderModuleCreateFailed	40	Vulkan shader module creation
InvalidVulkanDevice	41	Device pointer is null
VulkanNotInitialized	42	init_vulkan() not called
AlreadyWatching	60	Shader already registered
InvalidHandle	61	Shader handle not found
NotWatching	62	Shader not being watched
InotifyInitFailed	80	inotify_init1() failed
InotifyAddWatchFailed	81	inotify_add_watch() failed
WatcherThreadStartFailed	82	Thread creation failed

Error Handling

auto to_string(ShaderErrorCode code) -> std::string_view

Converts error codes to human-readable strings for logging.

Example:

auto result = shader_service.watch_shader(path, stage, callback, plugin);
if (!result) {
    LOGC_ERROR("Failed to watch shader: error code {}", result.error());
}

RAII Vulkan Usage

All Vulkan objects use vulkan_raii.hpp:

#include <vulkan/vulkan_raii.hpp>

// RAII shader module (automatic cleanup)
std::unique_ptr<vk::raii::ShaderModule> module;

// Creation
vk::ShaderModuleCreateInfo info{};
info.codeSize = spirv.size() * sizeof(uint32_t);
info.pCode = spirv.data();
module = std::make_unique<vk::raii::ShaderModule>(device_->createShaderModule(info));

// Raw handle extraction for pipeline creation
VkShaderModule raw = **module;

Benefits:

• No manual vkDestroyShaderModule calls
• Exception-safe
• Automatic cleanup on registry erase
• Memory leak prevention

Usage Example

#include <waxed/core/shader_service.h>

// Initialize
ShaderService::Config config;
config.cache_dir = "/path/to/cache/shaders";
config.enable_hot_reload = true;

ShaderService shader_service(config);
shader_service.init_vulkan(device);

// Watch a shader
auto callback = [](vk::raii::ShaderModule* new_module, const std::string& path) {
    LOG_INFO("Hot-reloaded shader: {}", path);
    VkShaderModule raw = **new_module;
    // Update pipeline cache
};

auto handle = shader_service.watch_shader(
    "/path/to/shaders/quad.vert",
    ShaderStage::Vertex,
    callback,
    "desktop"
);

if (handle) {
    // Get module for pipeline creation
    auto module = shader_service.get_shader_module(*handle);
    if (module) {
        vk::PipelineShaderStageCreateInfo stage_info{};
        stage_info.stage = vk::ShaderStageFlagBits::eVertex;
        stage_info.module = *module;
        stage_info.pName = "main";
        // Create pipeline...
    }

    // Later: cleanup
    shader_service.unwatch_shader(*handle);
}

Dependencies

• Vulkan-Hpp RAII: <vulkan/vulkan_raii.hpp>
• glslangValidator: Vulkan SDK’s GLSL reference compiler
• Linux inotify: Kernel file monitoring API
• C++26: std::expected for error handling

Future Enhancements

1. SHA-256 Hashing: Replace std::hash with cryptographically secure hash
2. Cache Pruning: Automatic cleanup of stale cache files
3. Include Directories: Support for #include directives in GLSL
4. Preprocessor Definitions: Pass defines to glslangValidator
5. Optimization Levels: Configurable -O flag for glslangValidator
6. Validation Layer Output: Capture and return GLSL compilation errors