Design Token Build Pipeline

A design token build pipeline automates the transformation of source token definitions into platform-ready outputs. Pipelines handle the mechanical work of parsing tokens, resolving references, transforming values, and generating files for each target platform. Well-designed pipelines produce consistent outputs reliably while providing feedback when issues occur.

What Is a Design Token Build Pipeline

A design token build pipeline is an automated process that takes token source files as input and produces formatted outputs for consumption. This includes reading source files, validating their structure, resolving token references, applying value transformations, and writing output files in appropriate formats.

Pipelines typically run as part of CI/CD systems, triggering automatically when token sources change. Local development may also invoke pipelines for preview and testing.

How Design Token Build Pipelines Work

A typical pipeline progresses through sequential stages:

Source Reading loads token files from the filesystem:

const sources = glob.sync('tokens/**/*.json');
const rawTokens = sources.map(file => ({
  path: file,
  content: JSON.parse(fs.readFileSync(file))
}));

Validation checks source tokens for correctness:

rawTokens.forEach(({ path, content }) => {
  const errors = validateSchema(content, tokenSchema);
  if (errors.length) {
    throw new ValidationError(path, errors);
  }
});

Reference Resolution follows token aliases to build the complete dictionary:

const resolvedTokens = resolveReferences(rawTokens);
// {color.action.primary} becomes the actual value

Transformation converts values for each platform:

const webTokens = transform(resolvedTokens, webTransforms);
const iosTokens = transform(resolvedTokens, iosTransforms);
const androidTokens = transform(resolvedTokens, androidTransforms);

Output Generation writes platform-specific files:

writeCSS(webTokens, 'dist/css/tokens.css');
writeSwift(iosTokens, 'dist/ios/Tokens.swift');
writeKotlin(androidTokens, 'dist/android/Tokens.kt');

Style Dictionary orchestrates these stages through configuration:

module.exports = {
  source: ['tokens/**/*.json'],
  platforms: {
    css: {
      transformGroup: 'css',
      buildPath: 'dist/css/',
      files: [{ destination: 'tokens.css', format: 'css/variables' }]
    },
    ios: {
      transformGroup: 'ios-swift',
      buildPath: 'dist/ios/',
      files: [{ destination: 'Tokens.swift', format: 'ios-swift/class.swift' }]
    }
  }
};

Key Considerations

• Pipeline failures should provide clear, actionable error messages
• Incremental builds improve performance for large token sets
• Parallel generation speeds up multi-platform output
• Output validation can catch transformation issues
• Version information should be embedded in outputs
• Clean builds should regenerate all outputs fresh
• Caching can improve repeated build performance
• Logging aids debugging and audit trails

Common Questions

How should pipeline stages be organized?

Pipeline stages should follow a logical progression where each stage completes before the next begins:

1. Clean: Remove previous build artifacts
2. Source: Read and parse token files
3. Validate: Check token structure and values
4. Resolve: Process references and aliases
5. Transform: Apply platform-specific conversions
6. Generate: Write output files
7. Verify: Check outputs for correctness

Each stage should be independently testable and produce clear success/failure status.

Hooks between stages enable customization:

StyleDictionary.registerHook({
  name: 'pre-transform',
  fn: (dictionary) => {
    // Custom logic before transformation
  }
});

Error handling at each stage should capture the failure context and report precisely what went wrong and where.

How should CI/CD integrate with build pipelines?

CI/CD integration ensures tokens build correctly and outputs stay current.

Pull request validation runs the pipeline on proposed changes:

on: pull_request
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm ci
      - run: npm run build-tokens
      - run: npm run test-tokens

Main branch builds generate outputs for consumption:

on:
  push:
    branches: [main]
jobs:
  build-and-publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm ci
      - run: npm run build-tokens
      - uses: actions/upload-artifact@v3
        with:
          name: token-outputs
          path: dist/

Scheduled builds catch drift from external dependencies:

on:
  schedule:
    - cron: '0 0 * * *'

Build status should block merges when pipelines fail. Clear failure messages help authors fix issues quickly.

How should multi-platform outputs be coordinated?

Multi-platform pipelines generate outputs for web, iOS, Android, and potentially other platforms from shared sources.

Shared stages handle common processing:

• Source reading (once)
• Validation (once)
• Reference resolution (once)

Platform-specific stages handle divergent needs:

• Transforms (per platform)
• Formatting (per platform)
• Output writing (per platform)

// Pseudo-code for coordinated multi-platform build
const dictionary = loadAndResolveTokens();

await Promise.all([
  buildPlatform(dictionary, webConfig),
  buildPlatform(dictionary, iosConfig),
  buildPlatform(dictionary, androidConfig)
]);

Parallel platform builds improve performance when platforms do not share output resources.

Cross-platform validation can verify consistency:

function validateCrossplatform(outputs) {
  // Verify same tokens exist in all platforms
  // Check value equivalence where expected
}

Summary

Design token build pipelines automate transformation from source definitions to platform-ready outputs. Pipelines progress through source reading, validation, reference resolution, transformation, and output generation stages. CI/CD integration ensures consistent builds and catches issues early. Multi-platform coordination shares common processing while handling platform-specific transformations in parallel. Well-structured pipelines produce reliable outputs while providing clear feedback when problems occur.