Automated accessibility testing setup with pa11y

Accessibility Testing with Pa11y

Pa11y is a CLI tool and Node.js library for automated accessibility testing. Unlike axe-core, Pa11y is better suited for batch audits of entire websites: it reads sitemap.xml and checks hundreds of pages sequentially.

Installation and Basic Usage

npm install -g pa11y pa11y-ci

# Check single page
pa11y https://example.com --standard WCAG2AA

# Output to JSON
pa11y https://example.com --reporter json > report.json

pa11y-ci: Batch Audit

// .pa11yci.json
{
  "defaults": {
    "standard":  "WCAG2AA",
    "timeout":   30000,
    "wait":      1000,
    "ignore":    [
      "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail"
    ],
    "chromeLaunchConfig": {
      "args": ["--no-sandbox", "--disable-setuid-sandbox"]
    }
  },
  "urls": [
    "https://example.com",
    "https://example.com/about",
    "https://example.com/contact",
    {
      "url":    "https://example.com/login",
      "actions": [
        "wait for element #login-form to be visible"
      ]
    }
  ]
}

pa11y-ci --config .pa11yci.json --threshold 5
# --threshold: acceptable number of errors (0 = strict mode)

Reading from sitemap.xml

# Automatically extracts URLs from sitemap
pa11y-ci --sitemap https://example.com/sitemap.xml \
  --sitemap-find "https://example.com" \
  --sitemap-replace "http://localhost:3000" \
  --threshold 0

Node.js API for Custom Reports

// scripts/a11y-audit.js
const pa11y   = require('pa11y');
const fs      = require('fs');

const PAGES = [
  { url: 'http://localhost:3000', name: 'Homepage' },
  { url: 'http://localhost:3000/catalog', name: 'Catalog' },
  { url: 'http://localhost:3000/checkout', name: 'Checkout' },
];

async function audit() {
  const results = [];

  for (const page of PAGES) {
    console.log(`Checking: ${page.name}`);
    const result = await pa11y(page.url, {
      standard:  'WCAG2AA',
      timeout:   20000,
      actions:   page.actions || [],
    });

    results.push({
      name:       page.name,
      url:        page.url,
      issues:     result.issues.length,
      critical:   result.issues.filter(i => i.type === 'error').length,
      warnings:   result.issues.filter(i => i.type === 'warning').length,
      violations: result.issues,
    });
  }

  // Save report
  fs.writeFileSync('a11y-report.json', JSON.stringify(results, null, 2));

  // Output summary
  console.table(results.map(r => ({
    Page:     r.name,
    Errors:   r.critical,
    Warnings: r.warnings,
  })));

  // Exit if critical errors found
  if (results.some(r => r.critical > 0)) {
    process.exit(1);
  }
}

audit();

Screenshots on Violations

const result = await pa11y(url, {
  screenCapture: `screenshots/${Date.now()}.png`,
  viewport: { width: 1280, height: 900 },
});

Pa11y vs axe-core Comparison

Timeline

Setting up Pa11y CI with sitemap audit and custom report: 1–2 business days.