Our HTTP alerting was in good shape. 500s, slow endpoints, error rates: all covered. Yet the incidents that hurt most kept coming from somewhere else:

  • The 02:00 invoice job never started, because the server restarted and the scheduler got stuck.
  • A queue consumer was up, but processed nothing for 20 minutes.
  • The bank statement file didn't arrive, so the import "succeeded" with 0 rows.
  • A worker loop deadlocked, and its exception was swallowed in a catch.

None of these has an HTTP status code. Ping-based monitors (Healthchecks.io, Cronitor and friends) cover part of it, but they add another SaaS, a ping URL in every job, and they can't tell "succeeded" from "succeeded with nothing done".

So I built RunSignal: cron and background-job monitoring on OpenTelemetry, with alerts as code, for .NET, Java, Go, Node.js, Python and Rust.

👉 GitHub: github.com/gokhanertogan/RunSignal (MIT)

How it works

  1. Record. An SDK records each run of a workload (success, failure, cancellation, duration), the items it handled, and heartbeats from long-running loops, as OpenTelemetry metrics. Every SDK uses the same metric names.
  2. Describe. Each workload declares what it should do: a cron schedule, a max duration, a heartbeat interval, a minimum output, a failure rate. The running app writes these to a JSON manifest.
  3. Provision. In CI, runsignal provision turns the manifest into alert conditions and a dashboard in New Relic, or runsignal export prometheus writes Prometheus alerting rules (plus a Grafana dashboard).

Alerts are evaluated by the backend, never in your process, because a process that has died cannot report that it is missing. Because the metrics are identical in every language, one CLI provisions alerts for a whole polyglot system.

The same idea in six languages

Where a scheduler exists, there is no code in the jobs: the integration records every run and registers the job with its cron at startup, so a job that never fires still gets a missed alert.

.NET: Hangfire (Quartz.NET works the same way):

builder.Services.AddRunSignal().AddHangfire(); builder.Services.AddHangfire((sp, c) => c.UseSqlServerStorage(cs).UseRunSignal(sp)); [Workload("nightly-invoice", MaxDuration = "30m", MinProcessed = 100)] public async Task GenerateInvoices(PerformContext? context = null) { /* ... */ }

Java: Spring @Scheduled (and Spring Batch), through Spring's own observations, with no proxies:

@Scheduled(cron = "0 0 2 * * *") @RunSignal(value = "nightly-invoice", maxDuration = "30m", minProcessed = 100) void generateInvoices() { ... }

Python: Celery tasks and beat schedules (APScheduler too), or a plain with block:

from runsignal import default_registry invoices = default_registry.workload("nightly-invoice", schedule="0 2 * * *", max_duration="30m", min_processed=100) with invoices.run(): for order in unbilled_orders(): create_invoice(order) invoices.processed()

Node.js: node-cron, croner, cron or node-schedule (BullMQ too):

import { cronJob } from '@runsignal/node/cron'; const invoices = cronJob('nightly-invoice', { schedule: '0 2 * * *', maxDuration: '30m' }, async (w) => { for (const order of await unbilledOrders()) { await createInvoice(order); w.processed(); } }); cron.schedule(invoices.schedule, invoices.run);

Go: robfig/cron, or Run with context cancellation and panic handling:

rscron.AddFunc(c, runsignal.Default, "nightly-invoice", "0 2 * * *", func(ctx context.Context) error { return generateInvoices(ctx) }, runsignal.MaxDuration(30*time.Minute))

Rust: sync and async runs; a dropped future is a cancelled run:

let invoices = registry.workload("nightly-invoice", Options::new() .schedule("0 2 * * *") .max_duration(Duration::from_secs(30 * 60)))?; invoices.run(|| generate_invoices())?;

The alerts

Alert Fires when
missed No success within the schedule plus grace
failed A run failed
overrun A finished run took longer than its max duration
hung A run is still running past its max duration
stalled No heartbeat in time (a loop that is alive but stuck)
low output Runs processed fewer items than expected on average
failure rate Too many runs or items failed
not reporting The service sent nothing at all

hung is new in 0.3.0. A duration histogram only sees runs that finish, so a run stuck forever never shows up there. Every SDK now reports the age of the oldest unfinished run as a gauge, so the alert fires while the run is still hanging.

Jobs with no SDK at all

Since 0.3.0, runsignal exec wraps any command (a shell script, a crontab entry, a container in a Kubernetes CronJob) as one monitored run:

runsignal exec --workload nightly-export --service billing -- ./export.sh
  • Outcome: exit code 0 is a success, anything else a failure with its exit code.
  • Signals: stop signals are forwarded and count as cancelled, and the command's exit code passes through.
  • Packaging: it's a 10 MB native executable with no runtime dependency. Copy it into any image with COPY --from=ghcr.io/gokhanertogan/runsignal.
  • Kubernetes: runsignal discover kubernetes writes the manifest straight from your CronJobs, including schedules, time zones and activeDeadlineSeconds.

Design choices

  • OpenTelemetry-native. These are plain OTel instruments, so the data goes to any OTLP backend. Alert provisioning targets New Relic and Prometheus today.
  • One contract, six SDKs. A shared conformance test defines the exact metrics a scenario must produce, and every SDK runs it in CI. An SDK that drifts fails its own build.
  • Cheap on the hot path. Recording a run doesn't allocate in .NET and Go (enforced by tests), and costs well under a microsecond in Rust and Node.
  • Idempotent provisioning. provision changes only what changed and never touches conditions it didn't create. Run it on every deploy.

Status

It's MIT-licensed and pre-1.0. Every alert kind has fired on a real New Relic account, and the Prometheus rules are covered by promtool test rules.