← back

nf-shard

Aug 2023

Open-source dashboard for monitoring Nextflow pipeline runs in bioinformatics analysis

https://github.com/gluonfield/nf-shard

nf-shard Launchpad - storing and submitting Nextflow pipeline runs from the UI
The Launchpad. Store pipeline definitions once, then submit runs against any registered compute environment without touching the CLI.

Motivation

Nextflow is the de-facto workflow engine in bioinformatics, but the moment you start running it across many samples, environments, or machines the experience falls apart. Seqera's enterprise nf-tower solves the monitoring side, but it's heavyweight, opinionated, and not open source.

nf-shard is a drop-in, self-hostable alternative that speaks the same nextflow.config snippet you already use. Drop in an endpoint, point your runs at it, and you get a UI for live status, historical runs, indexed search, tags, and workspaces. It is not trying to replace the full enterprise product. It's trying to be the smallest thing that makes day-to-day pipeline development sane.

Features

  • Live status of running tasks and per-process resource usage
  • Historical runs grouped into workspaces for separate projects and teams
  • Indexed search by workflow ID, run name, user, tag, project, and date range
  • Multi-condition (AND) search and freeform tags attached to each run
  • Slack integration for run start / completion / failure notifications
  • A Launchpad to store and submit pipeline jobs straight from the UI
  • nf-tower plugin-compatible API, so existing Nextflow configs work unmodified

Integration

To pipe a Nextflow run into nf-shard you only need to add a tower block to your nextflow.config. The accessToken can be any non-empty string in self-hosted mode.

tower {
  enabled     = true
  accessToken = "non-empty"
  endpoint    = "http://localhost:3000/api"
}

That's it. Run the workflow, open the UI, see the run appear in real time.

Stack

Next.js / React / TypeScript on the front, PostgreSQL with Prisma on the back, packaged as a single Docker image. The stack was picked deliberately for the largest possible contributor pool inside the bioinformatics and web communities.

Deployment

A single docker-compose --profile all up brings up Postgres and the server together. A one-click Vercel template handles the hosted route for teams that already use managed Postgres (Neon, Supabase, RDS). Container images are published to GHCR on every release.

← back