NodePipeline

A pipeline library that executes a graph of nodes described in YAML. Nodes are plain C# classes; a source generator produces an AOT-safe factory/validator for them at compile time — no reflection at runtime.

Main idea

            node class (C#)                    pipeline graph (YAML)
        ┌───────────────────────┐
        │ [NodeField(Input)]    │     nodes:
        │   NodeField<Image> In │◀────  - id: load
        │ [NodeField(Parameter)]│          type: FileImageSource
        │   NodeField<int> Size │        outputs: [Out]
        │ [NodeField(Output)]   │      - id: blur
        │   NodeField<Image> Out│────▶     type: GaussianBlur
        │ void Execute() { ... }│          parameters: {Radius: 50}
        └───────────────────────┘          inputs: {Input: load.Out}
                      │
        compile time  ▼
        ┌─────────────────────────────┐
        │ source generator            │
        │  · GeneratedNodeFactory     │  switch: YAML type → new Node()
        │  · GeneratedNodeValidator   │  [Required], ranges, defaults
        │  · DI extensions (optional) │
        └─────────────────────────────┘
                      │
        runtime       ▼
        YamlGraphLoader → PipelineRegistry.Register(...) → validation
                              │
                              ▼
        registry.Build(factory, id) → connected node graph → Execute()

Data flows only through connected ports (NodeField<T>), scalars arrive via parameters. The engine itself is minimal: topological order + Execute() loop.

Quick example

Define a node:

public class GaussianBlurNode : INode
{
    [NodeField(FieldDirection.Input)] public NodeField<ImageBuffer> Input { get; set; } = null!;

    [NodeField(FieldDirection.Parameter)]
    [HasValueBetween(0, 300)]
    public NodeField<int> Radius { get; set; } = null!;

    [NodeField(FieldDirection.Output)] public NodeField<ImageBuffer> Output { get; set; } = null!;

    public void Execute()
    {
        var src = Input.Value;
        var dst = new ImageBuffer(src.Width, src.Height);
        Convolution.Apply(src, dst, GaussianKernel.Create(Radius.Value));
        Output.Value = dst;
    }
}

Describe the pipeline in YAML:

name: pipeline-1
nodes:
  - id: load_image
    type: FileImageSource
    parameters:
      FileName: { value: "input" }
    outputs: [Output]
  - id: blur
    type: GaussianBlur
    parameters:
      Radius: { value: 50 }
    inputs:
      Input: { source: load_image.Output }

Register, build and run:

using var yaml = File.OpenRead("p1.yaml");
var config = new YamlGraphLoader().Load(yaml);

var factory = new GeneratedNodeFactory();       // generated at compile time
var validator = new GeneratedNodeValidator();
var registry = new PipelineRegistry(CultureInfo.CurrentCulture);

registry.Register(factory, validator, config.Name, config);
var pipeline = registry.Build(factory, config.Name);
pipeline.Execute();

With DI (NodePipeline.Engine.DependencyInjection):

services.AddPipeline();   // factory, validator, registry + all nodes with DI constructors

await using var scope = provider.CreateAsyncScope();
var registry = scope.ServiceProvider.GetRequiredService<PipelineRegistry>();
var pipeline = registry.Build(scope.ServiceProvider.GetRequiredService<INodeFactory>(), "pipeline-1");
pipeline.Execute();

Node naming

The YAML type: of a node is resolved to a class in two ways:

1. Explicit — [NodeType("crystal")] on the class: the attribute value is the node type.
2. Implicit — full type name (global:: and the Node suffix are trimmed), e.g. Nodes.FileImageSource for MyApp.Nodes.FileImageSourceNode.

Implicit names can be shortened with namespace prefixes: if the full name starts with a configured prefix, the prefix is trimmed too. Prefixes are configured in two places, which should be kept in sync:

• runtime — NodeNamePrefixSettings.SetNodePrefixes(["MyApp."]) at host startup (used by the generated factory when matching type:);
• compile time — the NodePipelinePrefixes MSBuild property (;- or ,-separated), read by the source generators and the IDE analyzer:

<PropertyGroup>
    <NodePipelinePrefixes>MyApp.;MyApp.Nodes.</NodePipelinePrefixes>
</PropertyGroup>

⚠️ Prefix trimming can merge distinct names (A.FooNode and B.FooNode become the same Foo) — a collision the runtime cannot warn about upfront. Duplicate type names are reported by the generators as NP0002 (error, across all referenced assemblies) and by the analyzer as NP1006 (warning, within the current compilation, implicit names included).

External resources

An input port can be bound to a named external resource instead of another node's output — use the resource: prefix in the connection source:

inputs:
  Photo: { source: "resource:logo" }

Pass an IPipelineResourceProvider at build time; the generated factory binds the value directly to the port. A required input whose resource is missing (or has an incompatible type) throws ResourceNotFoundException:

var pipeline = registry.Build(factory, config.Name, resources);

Resource connections create no graph edges and skip port-type validation at registration (the provider is only known at build time).

IDE analyzer

NodePipeline.Abstractions ships a Roslyn analyzer that catches node declaration mistakes while typing: NP1001 [NodeType] on a non-INode class, NP1002 [NodeField] on a non-NodeField<T> property, NP1003/NP1004 non-public node class or field, NP1005 validation attribute incompatible with the field type, NP1006 duplicate node type names (warning, implicit names included, NodePipelinePrefixes-aware), NP1007 redundant [DisallowNullableOutput] on a nullable field (warning). Duplicate names across assemblies are also reported as NP0002 by the source generators.

Packages

Package	Contents
NodePipeline.Abstractions	INode, NodeField<T>, attributes, validators
NodePipeline.Configuration.Abstractions / .Yaml	pipeline description model, YAML loader
NodePipeline.Engine	runtime (graph, connectors, executor) + source generator
NodePipeline.Engine.DependencyInjection	DI extensions + DI source generator
NodePipeline.Reporting.*	registration reporting abstractions / console reporter

Build & publish order: see PUBLISHING.md.