Skip to content

News

0.22.0 Metrics a scraper can actually read, Mod-Gearman without an open port, and a much smaller install

0.22.0 rebuilds the metrics side of the agent. /api/v2/openmetrics served the JSON keys verbatim โ€” names with dots, spaces and colons in them, no types, no help text, six significant digits โ€” which a strict parser rejected outright. It now serves a conformant, self-describing OpenMetrics document: every family named to the grammar, typed, described, carrying its unit, and per-instance readings collapsed into one family with a core, nic or drive label instead of one family per core. Every metric name changes, so read the upgrade note before you upgrade a scraped host.

The second theme is a new transport. GearmanClient makes the agent a Mod-Gearman worker: a Naemon or Nagios Core keeps scheduling its checks, the agent pulls them off a gearmand job server and answers them as native queries, and the monitored host needs no inbound port at all. The same module submits passive results back into the core’s result queue. It ships marked experimental and has not been exercised at scale: it is verified end to end against a real job server on both cores, but no one has yet run it across a large estate, so put it on a slice of your hosts before you move a whole Mod-Gearman installation onto it.

Then a long tail of security work โ€” the low-severity tier of the audit, in three parts, plus a second undefined-behaviour sweep and the ten worst threading bugs โ€” and a packaging release: common code moved out of every plugin into shared libraries, Windows ARM64 is back, and there is a Raspberry Pi package.

โœจ Highlights

  • ๐Ÿ“Š A conformant OpenMetrics exposition, and every metric name changes. Names follow the grammar (system.cpu.core 0.idle โ†’ system_cpu_idle_percent{core="0"}), every family carries # TYPE, # HELP and a # UNIT where the value is measured in something, counters are typed as counters, string readings come back as _info families, values keep their full precision and the body ends with # EOF. Per-core, per-NIC, per-drive and per-process readings become one family with a label, so sum by (core) has something to group on. The JSON endpoints, the dashboard, Graphite, collectd and Python submit_metrics report the same keys and values as before. openmetrics format = legacy reproduces the old body as a migration window. (#1533, #1535, #1538, #1539)
  • ๐Ÿ“ฅ New module: GearmanClient, a Mod-Gearman worker and result channel. Optional, off by default, and it speaks both flavours โ€” ConSol’s Mod-Gearman for Naemon and Nagios Enterprises’ Nagios-Mod-Gearman for Nagios Core 4.5+. mode = agent answers for the host it runs on; mode = proxy runs a whole hostgroup’s checks from one box. Windows and Linux. Experimental and not yet tested at scale โ€” try it on a few hosts first. (#1542)
  • ๐Ÿ”’ A settings source, include or attachment that is not https:// is refused. The remote store is the agent’s whole configuration, re-read at boot and on every housekeeping pass, and over plain http:// nothing authenticates the server. Opt back in per host with allow plaintext = true in boot.ini; every plaintext fetch is then logged as INSECURE. (#1529)
  • ๐Ÿ›ก๏ธ The audit’s low-severity tier lands, in three parts. A request may no longer weaken a credentialed target’s transport security or re-address submit_smtp mail; script arguments, check_docker’s endpoint and the remote-connection checks are confined; and the web server sends browser hardening headers, negotiates TLS 1.3 on Linux and stops minting a session token per request. (#1550, #1551, #1552)
  • ๐Ÿ”‘ check_tcp and check_ssh verify the server certificate by default. verify defaulted to none, so a TLS check handshook against any certificate at all and reported ok. It now defaults to peer with ca= falling back to the agent’s own trust bundle, and the checks gained certificate identity, SAN and STARTTLS-service keywords. (#1547)
  • ๐Ÿ” Agent-to-agent checking works, over the REST API, and the raw protobuf web API is gone. POST /query.pb let a caller write the header the core reads its identity from; its only consumer was NSCPClient, which had never actually worked. check_remote_nscp and friends now go through /api/v2/queries, authenticated. (#1552)
  • ๐Ÿงช New modules and check commands are marked experimental. A statement about stability, not about breakage: nscp test, the web UI, the REST API and the reference docs all say which checks may still change their options, keywords or output. (#1543)
  • ๐ŸŽจ The prompt and the web UI paint filter expressions. Both now read the check’s own vocabulary and colour a filter=, warning= or detail-syntax= as you type it โ€” a keyword the check does not offer is red before the check is ever run. A new GET /api/v2/queries/{query}/help is what the browser reads. (#1540, #1541)
  • โฑ๏ธ Reloads wait for the checks that are running. A settings reload holds new checks off a module while it applies the new configuration, a module can no longer unload or restart itself from inside a request it is serving, and connect and TLS handshake now count against a client’s configured timeout. (#1531)
  • ๐Ÿ“ฆ A fifth off the Linux install, and much more off Windows. Code every plugin compiled privately moved into shared libraries, and OpenSSL ships as DLLs instead of being linked into each of NRPE, NSCA, check_mk, the web server and the HTTP clients. (#1544, #1556)
  • ๐ŸชŸ Windows ARM64 is packaged again, and there is a Raspberry Pi package. NSCP-<version>-ARM64.msi/.zip are back (without PythonScript, and the ARM64 MSI does not bundle the VC runtime), and NSCP-<version>-debian-trixie-arm64.deb covers Raspberry Pi OS 64-bit and Debian 13 arm64. (#1546)

๐Ÿ” Detailed changes

๐Ÿ“Š OpenMetrics โ€” named, typed, described and labelled

The endpoint used to paste the JSON keys into the exposition verbatim. Names carried ., %, spaces and colons (system_mem_commited.avail, system_cpu_core 0.idle, disk_free_C:.total); there was no # TYPE and no # EOF; values were truncated to six significant digits, so 16 GB of memory scraped as 1.6554e+10; monotonic counts were typed as gauges, so rate() was unsafe on them; and string readings โ€” uptime, boot time, MAC address, power source โ€” were dropped entirely. A strict parser rejected the body, and the scenario page told you to repair the names with metric_relabel_configs.

Three things changed, and each of them renames families.

Names follow the grammar. % becomes the word percent, everything else outside [a-zA-Z0-9_] becomes _, runs collapse, and a name that would not start with a letter borrows a metric_ prefix.

JSON key Metric name
system.mem.physical.% system_mem_physical_percent
system.cpu.core 0.idle system_cpu_core_0_idle
disk.free.C:.total disk_free_C_total

Every metric carries a description, a type and a unit. # HELP on every built-in family, # UNIT wherever the value is measured in something โ€” which renames the family again, since a family that declares a unit has to end in it (system_mem_physical_total โ†’ system_mem_physical_total_bytes). Counters are typed as counters and carry the reserved _total suffix on their sample; strings come back as an _info family in the node_uname_info shape.

Per-instance metrics are one family with a label.

# TYPE system_cpu_idle_percent gauge
system_cpu_idle_percent{core="0"} 93
system_cpu_idle_percent{core="1"} 91
system_cpu_idle_percent{core="total"} 95

The label is core, cpu, nic, zone, battery, exe, disk, drive or pdh_instance depending on the bundle โ€” pdh_instance rather than instance, because Prometheus attaches its own instance label to every sample. Windows and Linux spell a CPU core differently in the JSON key (core 0 and core_0); neither spelling reaches the label, which is the bare 0 on both, so one query works across a mixed fleet.

Around that:

  • /api/v2/metrics?meta=1 serves the same keys and values with their help text, unit, type and labels under a metadata object. Without meta the endpoint is byte for byte what it was.
  • GraphiteClient can send the labels as carbon tags (metric tags = true, off by default โ€” a carbon older than 1.1 stores path;core=0 as the name).
  • CollectdClient mappings can read the labels and the types. A variable set to label:core expands to every value of that label instead of a regular expression over flat keys, and auto: sends whatever the producing module declared a counter as a DERIVE and everything else as a GAUGE.
  • Predefined PDH counters can describe themselves with help and unit keys, and Python fetch_metrics accepts a dict per value ({"value": 42, "help": "โ€ฆ", "unit": "bytes", "type": "counter"}, plus "labels").
  • Windows only: system.mem.page.% and system.mem.physical.% report different numbers, because they were reporting the wrong thing โ€” both divided the commit charge by the commit limit. Alert thresholds tuned against the old reading need re-checking.

๐Ÿ“ฅ GearmanClient โ€” Mod-Gearman, either flavour

A Naemon or Nagios Core installation running Mod-Gearman keeps its scheduler, its check definitions and its escalations; the agent registers for the queues you name, pulls jobs off gearmand and answers them as native NSClient++ queries. Nothing listens on the monitored host.

Setting Effect
mode = agent Answers only for the host it runs on; a job for another host_name is answered UNKNOWN
mode = proxy Answers every check on the queues it registered โ€” one Windows box running a whole hostgroup through check_nrpe, check_wmi and the rest

The same module submits passive results into the core’s result queue (/settings/gearman/client, channel GEARMAN, command submit_gearman), which is what lets a Mod-Gearman installation drop NSCA. The two halves are independent.

Two things to unlearn when writing the check_command on the core, since neither fails in a way that names the cause: write host=$HOSTADDRESS$, not -H $HOSTADDRESS$ (a two-character first token puts the argument parser into key-value mode and the check answers with a help screen), and warning=load gt 80, not warning=load>80 (> is a metacharacter, and allow nasty characters is false by default). See Mod-Gearman for the full setup on either core.

The module is marked experimental, and that mark is doing real work here: its settings, queue handling and output may still change, and while it has been run end to end against a real gearmand on both cores, it has not been tested at scale โ€” not against hundreds of hosts, a deep job backlog, or a proxy answering for a large hostgroup. Roll it out to a slice of the estate first and keep the existing transport until you are satisfied. Reports of how it behaves on a real workload are exactly what the experimental mark is asking for. (#1542)

๐Ÿ›ก๏ธ Security

The low-severity tier of the audit landed in three parts, alongside a second undefined-behaviour sweep and the settings/web/build hardening from #1529.

Outbound clients. The request-override guard now covers the keys that decide how a connection is protected โ€” verify, insecure, no-psk, ssl, tls-version, ca, certificate, allowed-ciphers, dh โ€” and, for SMTP, recipient and sender (behind their own allow recipient override). proxy= and no-proxy= count as moving the request, since a caller-chosen proxy receives it whole with the configured NRDP token in it. payload-length is clamped to what each protocol accepts, 65536 for NSCA and 1 MiB for NRPE. (#1526, #1550)

Scripts and check targets. A NUL in a script argument is refused โ€” CreateProcessW reads the command line as a C string, so it truncated there and silently dropped every operator-fixed argument after the substitution point. ext-scr add --import reads only from the script folders. A script now receives only its own stdin and its own stdout/stderr pipe ends instead of every inheritable handle of the service. host= on a docker check must match the configured endpoint. And on Linux, a script section with user, domain or password set is refused rather than run as the service account: those keys are implemented by the Windows launcher only, so a script sandboxed with user = nobody was not sandboxed at all. (#1523, #1551)

The web server. Every response carries X-Frame-Options: DENY, X-Content-Type-Options: nosniff, Referrer-Policy: no-referrer and a content security policy, plus Strict-Transport-Security over TLS โ€” so a page that embeds the agent’s UI in a frame will stop working, which is the point. [/settings/WEB/server] gains tls version (default 1.2+) and allowed ciphers. Logging out calls DELETE /api/v2/login, so the bearer token stops working immediately; the in-memory log buffer is capped at 1000 entries, where an unauthenticated peer could previously add one per rejected request for the life of the process; and nscp web install-ui no longer stages its download in ${temp}. (#1552)

Elsewhere. The shared /settings/default/password is registered sensitive by the core rather than by whichever server module happens to be loaded, so every agent redacts it. Elastic and Op5 submissions log when credentials go over an unverified https link, as the Icinga, NRDP and NRPE clients already did โ€” and the Icinga warning, which repeated on every submission, is logged once per target now. allowed hosts finally understands the * ranges it has always advertised (192.168.1.*, 10.*, a bare *); such an entry used to throw out of the address parser and take the whole listener with it. The Windows release build pins every third-party action to a commit and verifies every download against .github/dependency-checksums.txt. (#1529, #1532,

1534, #1537)

Fleet bundle signatures changed shape, and this one needs action. An Ed25519 signature used to cover the bare SHA-256 digest of a bundle’s bytes, which bound nothing about which bundle those bytes were: an old signed blob could be re-advertised under a new id, name or version and still verify. It now covers a canonical descriptor of the bundle’s identity, which means the fleet server must be upgraded before โ€” or at the same time as โ€” the agents. A mismatch is safe but inert: the agent logs signature verification failed, keeps the configuration it last applied, and stops picking up changes. (#1536)

๐Ÿ” Agent-to-agent checks over REST

POST /query.pb and POST /settings/query.pb are removed. The first handed the core a message whose header the caller wrote, and the core reads the calling module and user out of that header. The second had been unreachable for several releases. Nothing that talks to an agent over HTTP used them: Icinga’s check_nscp_api asks for GET /query/{name}, untouched, and the web UI uses /api/v2.

Their only consumer was NSClient++ itself, through NSCPClient โ€” and that never worked: it passed the serialized message as the HTTP request target, so every request was malformed, and the configured password was never sent. check_remote_nscp, remote_nscp_query and remote_nscpforward now run the command through GET /api/v2/queries/{command}/commands/execute. If you have an NSCP target configured:

Setting Was Now
path /query.pb /api/v2/queries
TLS off unless ssl = true on unless no ssl = true
password read, never sent sent; the remote’s admin user needs queries.execute

exec_remote_nscp and submit_remote_nscp are removed rather than ported: both posted an NRPE-style string to the protobuf route, which the remote parsed as an empty message and answered with nothing โ€” and the submit path reported success regardless. Use NSCA, NRDP or another submit client for passive results. (#1552)

๐Ÿงช Experimental modules and commands

A module or check command that is new enough that its options, filter keywords and output may still change now says so: nscp test appends (experimental) in queries, aliases, list and plugins and shows a Status: line in desc, the web UI shows a chip, the REST API reports an experimental field, and the reference documentation renders a marker in the command tables and a note on the command. The recently added check_* commands of CheckDisk, CheckDocker, CheckNet, CheckNSCP and Scheduler are marked, as are the CheckMSSQL, CheckMySQL, CheckSecurity, CheckWindowsApps, GearmanClient and NSCANgClient modules in full. Out-of-tree modules declare it in module.json. (#1543)

๐ŸŽจ Filter expressions, painted

The prompt already knew a command name from a typo. It did not know anything about what came after the =, which is where the mistakes actually are: filter=fre < 10% is a check that runs, matches nothing and reports OK. The value of every option that takes a filter expression or a syntax template is now read as what it is, following the where grammar and the placeholder rules the engine itself uses โ€” a keyword the check offers is painted, one it does not is red, before enter is pressed. Filter functions, operators and number+unit literals each get their own colour.

The web UI’s arguments field does the same, from the same registry data: GET /api/v2/queries/{query}/help returns one check’s whole vocabulary โ€” every option with its default, flags and description, and every filter keyword it offers. An alias declares no keywords of its own, so the endpoint follows it to the command it stands for and says which one in keyword_source. (#1540,

1541)

โฑ๏ธ Threading, reloads and timeouts

The ten highest-severity findings of the concurrency audit are fixed, and several of them are visible from outside:

  • A settings reload holds new checks off a module while its loadModuleEx applies the new configuration, and waits up to five seconds for checks already inside it to return. A reload is no longer instantaneous on a busy agent; if a check holds a module longer than that, the reload proceeds anyway and logs which module it was.
  • Unloading the module that is serving the request is refused instead of taking the agent down with it โ€” in practice POST /api/v2/modules/WEBServer/commands/unload. Reloading a listener from inside a check that the same listener is serving is refused too, instead of leaving that listener dead.
  • Configuration downloaded over HTTP gives up on a read or write that stalls for 30 seconds, instead of waiting indefinitely, and no longer holds the settings instance lock while it downloads.
  • Submissions over NRPE, NSCA, NSCP and check_mk apply the configured timeout to connecting and to the TLS handshake, not just to the exchange.

Underneath: the Lua script manager and the collector pointers are published atomically so a reload cannot free them under a running check, the scheduler watchdog no longer re-arms the pool during shutdown, a socket server refuses to be stopped from one of its own threads, the Python function registry is read under the GIL, and a CheckEventLog filter object no longer closes a handle it does not own. (#1531, #1534)

๐Ÿ› Bug fixes

  • check_nt FILEAGE checks one file instead of a directory of them. It mapped onto check_files, which walks a whole tree, so a directory argument reported whichever file the walk happened to emit first โ€” not the oldest, not the newest, just arbitrary. It maps onto check_single_file now; a directory fails with an error.
  • The Windows installer no longer rewrites NRPE transport security it did not configure. The MSI recognised only insecure = true and verify mode = peer-cert, so a listener running TLS without client certificates, or one with a hand-written cipher string, fell through and got a preset applied over it. (#1562)
  • nscp nrpe install writes use ssl, not ssl = true โ€” a key the server never reads. On a host where use ssl = false had been set previously, the command claimed encryption and client-certificate authentication while the listener stayed in plaintext. The legacy cipher default also drops its !ADH, which never excluded the anonymous elliptic-curve suites.
  • The PDH counter browser narrows on every filter and ignores case. --list, --filter and --counter shared one variable, so whichever came last won and the others were silently discarded; --filter is repeatable now. Matching is case insensitive, so --list disk finds what --list Disk finds.
  • A round-robin counter with a zero or unparseable buffer size is refused and named in the log, instead of being loaded with a buffer that holds nothing.
  • A collectd value list naming a metric the snapshot does not carry is no longer sent as a zero โ€” it reported a measurement nobody took. Such a value list is skipped in whole, since a collectd value list is positional.
  • check_ping keeps listening for its own echo reply instead of giving up on the first reply that arrives, an HTTP status line with a reason phrase parses, every line of a remote check result is kept, and a string Python cannot encode no longer crashes the agent.
  • check_dns’s host= description said the wrong thing. (#1545)

๐Ÿ“ฆ Packaging

  • Shared runtime libraries. nscp_net.dll (sockets and TLS), nscp_client.dll (the sender modules’ shared command line), nscp_json.dll and OpenSSL as libcrypto-3-x64.dll / libssl-3-x64.dll now ship next to nscp.exe, one copy for the whole service, where before NRPE, NSCA, check_mk, the web server, the HTTP clients and the checksum checks each carried their own. plugin_api.dll absorbed the settings and program-options helpers. About a fifth off the install on Debian and RedHat, and more than that on Windows. The legacy XP build still links everything statically. (#1544, #1556)
  • Windows ARM64 is back as NSCP-<version>-ARM64.msi and .zip. Two differences from x64: no PythonScript (the package is cross-compiled and there is no ARM64 CPython to embed), and the MSI does not bundle the Visual C++ runtime, because Microsoft ships no ARM64 merge module for this toolset โ€” install vc_redist.arm64.exe first on a fresh machine. (#1546)
  • A Raspberry Pi OS package, NSCP-<version>-debian-trixie-arm64.deb, built on Debian 13 for Raspberry Pi 3 and newer and for Debian 13 arm64 in general. 64-bit only, and without the managed (C#) plugin API, since Debian does not package the .NET SDK. (#1546)
  • The web bundle is built with npm ci and gated on npm audit, so the bytes in the web zip and in the MSI’s web/dist match package-lock.json. Pull requests build only the newest RedHat, and a failed dependency download fails at the download instead of somewhere later.

โš ๏ธ Upgrade notes

  • Every OpenMetrics family name changes. Drop any metric_relabel_configs block that rewrote dots to underscores โ€” the agent does that itself now. Update dashboards, recording rules and alerts: system_cpu_core 0.idle is system_cpu_idle_percent{core="0"}, disk_free_C:.total is disk_free_total_bytes{drive="C:"}, workers_jobs is workers_jobs_total. Exclude core="total" from anything that aggregates over cores. If dashboards cannot be updated first, set openmetrics format = legacy under [/settings/WEB/server] as a migration window โ€” it is deprecated and will be removed.
  • Upgrade the fleet server before or with the agents. Bundle signatures now cover the bundle’s identity, not only its bytes. A version mismatch is inert rather than damaging: the agent keeps the configuration it last applied and stops picking up changes.
  • A settings source, include or attachment over plain http:// is refused. Move it to https://, or set allow plaintext = true under [tls] in boot.ini per host.
  • check_tcp / check_ssh against an internal or self-signed service now fail with tls_handshake_failed. Point ca= at the issuing CA or add verify=none. sni= on a non-TLS connection is now rejected rather than ignored.
  • POST /query.pb is gone. If you have an NSCP client target, remove an explicit path or point it at /api/v2/queries, expect TLS unless you set no ssl = true, and give the remote’s admin user the queries.execute grant. exec_remote_nscp and submit_remote_nscp are removed.
  • Requests may no longer weaken a credentialed target’s transport security (verify, insecure, ssl, ca, tls-version, proxy, โ€ฆ) or re-address submit_smtp mail. Supply the credentials with the request, configure the variant as its own target, or set allow host override = true / allow recipient override = true.
  • External scripts with user, domain or password set are refused on Linux โ€” those keys were never implemented there. Put the identity change in the command itself with sudo.
  • The web UI cannot be framed any more, and check_nt FILEAGE naming a directory now fails instead of reporting an arbitrary file’s age.
  • A hand-rolled deployment must copy the new shared libraries from the installation root alongside modules\*.dll, the OpenSSL DLLs included. Installing from the MSI or the Debian/RedHat packages needs nothing.
  • On Windows ARM64, install the VC++ ARM64 redistributable before the agent on a fresh machine, and use the x64 package under emulation if you need PythonScript.

Security notices for this release are on the security notices page, and the full list of behaviour changes is on the upgrading page.

Download

You can download the new version from GitHub

// Michael Medin

Thank you, Nagios World Conference 2026

Nagios World Conference 2026 wrapped up in the Twin Cities today, so first of all: thank you. To everyone who came to my two sessions, asked questions, or said hi.

Thanks also to the Nagios event team. I genuinely believe that this is one of the best conferences in the world. And you proved that you can do this consistently. Great talks, great venue, great catering and excellent evening events.

The slides

Both decks are up:

A mod-gearman proof of concept

The first thing I am bringing home is a working Mod-Gearman proof of concept in NSClient++. I hacked it together this week to find out if it was even doable. I stayed on Swedish time, so I woke up at 1 am every morning coding all the way till 7 am.

Gearman had not been on my radar for years. What changed is not that I finally got around to it, but that the missing piece showed up on its own, as a side effect of other work. NSClient++ now ships close to a hundred checks (thats on par with monitoring-plugins), and a good number of them are network checks: check_http, check_tcp, check_ping, check_dns, check_ssh, check_ntp_offset, check_certificate, check_mysql, check_docker and so on. All of them run inside the agent. No forking, no plugin process per check, no interpreter start-up.

That matters for a Gearman worker. A normal worker pays for a process launch on every job it picks up. An agent that can already talk NRPE, NSCP, HTTP, TCP, DNS and ICMP in-process does not have that cost. Sitting in talks this week, listening to how people distribute their check load, it finally clicked that nothing was stopping the agent from being the worker. Well, I guess I need to take another stab at check_by_ssh.

So I tried it, and it works. NSClient++ can pull jobs off a Gearman queue and hand the results back. In practice that means the agent could take your whole check load if you wanted it to, on Windows and Linux, local checks and remote ones alike, without a separate worker fleet for the non-Windows part.

Nagios showing service checks served by NSClient++ over mod-gearman

That is a real Nagios in my lab, and every service on nscp-lab in that list was executed by NSClient++ after being picked up from the Gearman queue. Nagios never talks to the agent. It puts jobs on the queue, and the agent takes them off.

The details I was most pleased with are the boring ones. The states are real states, not just “it replied”: Disk C and Memory come back CRITICAL with their performance data intact, so thresholds and graphs still work. Missing command returns UNKNOWN with Unknown command(s): check_no_such_thing, so a failure comes back through the queue as a failure instead of disappearing or hanging. A passive result submitted by the agent shows up in the same list. And after close to five hours the queue is still empty, 0 jobs running and 0 jobs waiting, which is what you want from a worker that keeps up.

This is a proof of concept. It is nowhere near a release and I am not promising anything yet. But going from “have not thought about this in years” to a worker pulling jobs off a queue in a week, because of something I heard in a conference room, is a pretty good argument for going to conferences.

The AI talk

The second thing I am bringing home is less about code.

I sat in on Transforming Nagios with AI: Building an LLM-Powered Monitoring Assistant and Predictive Anomaly Detection Engine by Sunil Kathait from Ellucian. What stuck with me was not the model or the prompting. It was what the assistant needed in order to be useful, and what it was actually for.

The number that matters is mean time to recovery: how long from something breaking to it working again. If you break that time down, most of it is usually not the fix. Restarting a service takes thirty seconds. Working out that a service needs restarting, which one, on whose box, and whether you are even the right person to be looking at it, is where the time goes.

So the faster you understand what is wrong, the faster you recover. And understanding is mostly a context problem. That made me realize that context is the piece that has always been missing from classic Nagios-style monitoring, with or without AI. And the one thing that graph based monitoring always had.

We have spent decades getting good at alerting. DISK CRITICAL - C:\ used 94% is accurate, cheap and reliable. It tells you something is wrong and starts the clock. What it does not tell you is whose machine that is, what runs on it, whether it matters at 3 AM, whether it has done this every month-end for two years, what someone did about it last time, or who to hand it to. The on-call engineer answers those questions from memory, a wiki and a Slack search, and every one of those minutes is recovery time.

This is where an LLM actually earns its place. Taking a pile of context and turning it into “here is what is probably wrong and what to try first” is what these models are good at. Given the alert plus what the machine is, what it does, what it has done before and what fixed it last time, a model can give you a decent first guess in seconds, at 3 AM, for someone who has never seen the host before.

But it only works in that order. Give the model nothing but the alert string and all it can do is rephrase the alert string. There is nothing to reason about.

So the AI story and the “classic monitoring is missing something” story turn out to be the same story. Both need context. Get that in place and your people recover faster today, and an LLM has something worth reasoning about later.

Thanks to Sunil for the session. It was the most useful hour of my week.

Where that leads for NSClient++

If context is the missing piece, the agent is a natural place to collect a lot of it. It is already on the box, and it already knows things nobody wrote down. Every question it can answer is one the person on call does not have to go and chase.

There is a small version of this in NSClient++ today: tags, name = value facts an agent reports about its own host, such as drives, os_name and os_version. The loaded modules contribute them, and the web interface and newly launched fleet server read them. Right now they are mostly a cheap inventory. I think they are the seed of something more useful.

What I want to explore is extending the fleet side to hold real context about every host it knows about:

  • Context from the host itself. Not just the OS, but what the box appears to be: which roles and products are installed, what is listening, which services matter here. The agent can work most of that out without anyone maintaining a spreadsheet.
  • Context from you. Owner, team, environment, criticality, escalation path, a link to the runbook. Set once through the fleet, instead of copied into every check definition.
  • Context from history. The agent has seen this disk fill and drain a hundred times. “This is the fourth time this week” and “this happens every month-end” are two of the most useful sentences you can put in an alert, and neither needs a model.
  • Context from the moment it broke. This is the one I find most interesting. The agent is the thing that sees the check fail, and it is on the box when it happens, so it can grab a snapshot right then: current metrics, what CPU, memory and disk queues were doing, what was running, what the event log just said. Nagios learns that a check went critical. The agent can know what the whole machine looked like in that second.

The context should not travel with the alert

The obvious design here is the one I think is wrong.

The tempting thing is to attach the context to the check result and ship it along, so every alert carries its own explanation. I do not want to do that. The alert should stay what it is: small, cheap and unchanged, exactly the thing NRPE, NSCA and Nagios already understand. These checks run constantly and almost all of them come back OK. Shipping a host profile along with every OK result, to repeat things that have not changed since last Tuesday, is a lot of traffic for no new information.

Instead, the alert manager does the joining. Something breaks, the alert arrives as it always has, and at that point, when it is finally worth knowing, the alert manager asks the fleet server about the host: what is this box, who owns it, what runs on it, and what did it look like when the check failed. Then it either renders something useful for the person on call, or hands the lot to an LLM and asks for a suggestion.

The one thing that cannot wait is the snapshot. Who owns a box and what runs on it will still be true in ten minutes, so there is no hurry, ask when you need it. The state of the machine at the instant the check failed will not be true in ten minutes. It is gone. So the agent has to capture it the moment it sees the failure and hold on to it. Capture at the moment, fetch on demand. By the time a human has read the notification, opened a console and logged in, the spike has flattened out and the process that ate the box has exited. That is why “it looks fine now” is such a familiar way for an incident to end.

What I like about this approach:

  • Nothing changes on the wire. No fatter payloads, no changes to Nagios, NRPE or NSCA. Your existing pipeline keeps working as it does today.
  • Context is fetched when something is actually wrong, not on every check of every host for the 99% of the time everything is fine.
  • The context is current. You get what is true when you ask, not whatever was true when the check happened to run.

So the interesting work is not really in the alert path. It is in giving the fleet server something worth asking about. But we shall see what happens, currently this is just an idea…

Thanks again, everyone. And I hope to see you next year!

// Michael Medin

NSClient Fleet 0.1.0 โ€” central configuration management for NSClient++

NSClient Fleet is a new, separate product: a server that holds the configuration for a whole estate of NSClient++ agents, hands each host the part that applies to it, and reports what every host is actually running. The first release is out.

Until now there have been two ways to configure a large number of agents. You can point each one at an ini file on a web server and let it pull the file (CONFIGURATION_TYPE=), or push settings to each agent over the REST API from whatever orchestration you already run. Both work, and both remain supported. Neither keeps track of the estate, though: the only inventory is the one you maintain by hand, so questions like which version of check_backup.ps1 is on which host or which hosts have not checked in since Tuesday have nothing to answer them.

Fleet turns that around. Agents enroll themselves and report what they are, so the server builds the inventory instead of consuming one. Grouping by reported facts, configuration assembled per host, and drift reported as a comparison rather than a guess all follow from that.

It lives in its own repository and is entirely optional โ€” NSClient++ works exactly as it always has without it.

โœจ Highlights

  • ๐Ÿท๏ธ Self-enrolment and self-describing hosts. A few extra MSI properties and the host appears in the inventory with its OS, version, agent build, drives and detected roles, as tags reported by the agent rather than typed in.
  • ๐ŸŽฏ Groups select hosts by tag; bundles attach to groups. A host that starts reporting sqlserver=detected falls into the SQL group and picks up the SQL monitoring bundle on its next poll.
  • ๐Ÿงฉ Bundle templates that edit real INI. The forms are bound to actual INI keys and write back with line edits, so the INI text stays the source of truth and hand-edits survive.
  • ๐Ÿ” Encrypted bundles. Sealed in the browser with AES-256-GCM before upload; the server stores an 8-byte key fingerprint and delivers a blob it cannot read. The agent option require encrypted bundles makes it refuse anything unsealed.
  • ๐Ÿ“Š Host status derived from reported hashes. In sync, out of sync, offline and lost, plus a local config flag for hosts carrying local settings that outrank what the server sends.
  • ๐Ÿ”Œ One port. 443 serves the operator UI, the agents and certificate issuance, separated on the TLS handshake by ALPN.
  • ๐Ÿ“ฆ One static binary and a SQLite file. No runtime and no database server; Linux and Windows, x86-64 and arm64, or a container image.

What it takes over

A management server is one more thing to run, patch and be woken by, so it is worth being clear about what comes out in exchange. In a typical config-file setup, Fleet replaces:

  • the orchestration that pushed configuration
  • the web server and the directory layout behind it
  • the include-file naming convention
  • the inventory spreadsheet
  • tracking which script is deployed where

Installation and enrolment

Installing an agent into a fleet uses the same MSI command line as before, with different properties:

msiexec /qn /i NSCP-<version>-x64.msi ^
    FLEET_SERVER=https://fleet.corp.example ^
    FLEET_TOKEN=<one-time token> ^
    FLEET_BUNDLE_KEY=<your key>

Where the config-file approach pointed at an ini URL, this points at a server and carries a one-time token, so moving between the two is a change of MSI properties rather than a migration.

The token is single-use and short-lived. The agent generates its own keypair, sends a certificate request, and receives a client certificate along with everything it needs to trust the server; the token is then spent and will not be accepted again. After enrolment there is no token anywhere โ€” the certificate is the identity.

sequenceDiagram
    participant A as Agent
    participant F as Fleet server
    A->>F: POST /enroll/v1 โ€” one-time token + CSR
    F-->>A: client cert ยท CA ยท pinned server cert ยท signing key
    Note over A,F: everything below is mTLS, always agent-initiated
    A->>F: GET /agent/v1/desired-state?current_hash=โ€ฆ
    F-->>A: 304 โ€” nothing changed, sleep N
    A->>F: GET /agent/v1/desired-state?current_hash=โ€ฆ
    F-->>A: 200 โ€” bundle set + state hash
    A->>F: GET /agent/v1/bundles/:id
    A->>F: POST /agent/v1/state-report

Every exchange after enrolment is initiated by the agent, so there is no inbound connection to a monitored host, no port to open and nothing pushed. Most polls return 304.

Hosts describe themselves

Agents already know a good deal about the machine they run on, and now report it as tags:

Tag Where it comes from
os the agent
os_name Windows Server 2022 โ€” for reading
os_version 10.0.20348 โ€” for matching
nscp_version which agent build is on the host
drives c:,d: โ€” from CheckDisk
sqlserver detected โ€” read from the registry, so a stopped instance still counts
operator-defined any service or unit mapped to a tag: MSSQLSERVER=sql-server

Scripts can publish tags too, which covers the facts specific to a given estate: whether IIS is listening on 443 (role=web-frontend), which in-house application is deployed (app=billing, app_version=4.2), or whether a provisioning marker file exists (env=lab). A script-reported tag behaves exactly like a built-in one in groups and selectors. The agent sends its full tag set on every report, so a tag that stops being reported simply disappears โ€” there is no cleanup job and no stale inventory.

Because these are the host’s own claims about itself, a group only matches on agent-reported tags if it is built to do so. Reported facts are meant for grouping, not access control; anything gating access to secrets should match on operator-set tags instead.

Groups and bundles

flowchart LR
    T["tags<br/>os=windows<br/>sqlserver=detected"] --> G(["group ยท SQL Servers<br/>sqlserver exists"])
    G --> B1["sql-monitoring 2.1.0<br/>priority 100"]
    G --> B2["windows-base 1.4.0<br/>priority 10"]
    B1 --> C(["merged config<br/>+ scripts"])
    B2 --> C
    O["this one host<br/>priority 1000"] --> C

A group is a rule against tags rather than a list of machines โ€” sqlserver exists, os is in a given set, this and not that. Groups are built in the UI rather than typed as a query, so there is no expression language to inject into.

Bundles attach to groups, not to hosts, and each attachment carries a priority. Priorities are layers: windows-base at 10 underneath, sql-monitoring at 100 on top. Later layers win, and a bundle can delete a key set by the layer below. For cases that genuinely concern a single machine, a host override sits above every group at priority 1000.

Bundles

A new bundle starts from a template. The set is derived from the scenario guides โ€” Windows and Linux server health, disk space, services and processes, performance counters, real-time alerts, SQL Server and network checks โ€” plus one per delivery mechanism (NRPE, Checkmk, Prometheus, NSCA, NSCA-NG, Icinga 2, NRDP, Graphite and scheduled baselines).

Two things about how they are built are worth knowing.

Each template covers one concern. Check templates define transport-neutral aliases and say nothing about how results leave the host; each transport is a separate bundle. A group therefore gets “Windows health” and “deliver over NRPE” as two layers, and changing monitoring system means swapping one bundle rather than editing checks.

The form is a view of the INI, not a generator. Every field is bound to real INI keys and the INI text remains the source of truth: the form reads values out of the text and writes back with line edits, so raw editing, comments and hand-written sections survive a round trip. A bundle remembers which template produced it, so reopening it later gives the form back rather than unstructured INI.

Bundles can also be built without the UI, since a bundle is a zip file:

windows-base-1.4.1.zip
โ”œโ”€โ”€ bundle.toml        name = "windows-base" ยท version = "1.4.1"
โ”œโ”€โ”€ config.json        the config fragment โ€” JSON Merge Patch (RFC 7396)
โ””โ”€โ”€ scripts/
    โ””โ”€โ”€ check_backup.ps1
curl https://fleet.corp.example/api/bundles \
  -H "Authorization: Bearer nsk_โ€ฆ" \
  -F name=windows-base -F version=1.4.1 \
  -F bundle=@windows-base-1.4.1.zip

The config fragment is a standard JSON Merge Patch โ€” objects deep-merge, scalars replace, null deletes โ€” and the agent renders the merged result to NSClient INI on the host. Layering is therefore a property of the format rather than of the editor, and a hand-built bundle participates in it exactly like a template-built one. Upload is an API key operation, so bundles can live in git and be built and published by CI.

Encrypted bundles

Bundles carrying credentials โ€” SQL Server connection strings, passive transport passwords โ€” can be encrypted in the browser before upload, with AES-256-GCM and a key the server never receives. The server stores an 8-byte key fingerprint, enough for the UI to report a key mismatch, and otherwise signs, stores and delivers a blob it cannot read.

The bundle’s name and version are authenticated into the ciphertext, so a compromised server cannot serve one validly encrypted bundle in place of another; a mismatch fails decryption. Keys reach agents at install time, over the same out-of-band channel as the enrolment token.

The agent option require encrypted bundles makes an agent refuse anything unsealed. Since only a key holder can produce a bundle that decrypts, a server facing agents in that configuration cannot deliver configuration or scripts of its own making.

The trade-off is that there is no escrow: recovery would require the server to hold the key. A lost key means re-uploading the affected bundles under a new one. Rotation is supported โ€” agents hold a key list, newest first, and select by fingerprint, so bundles encrypted under the previous key keep working during a rotation.

Host status and drift

Status Meaning Where it comes from
in sync running exactly what the server would send it the hash the agent reported matches the hash the server would serve now
out of sync a change is pending, or an apply failed hashes differ; an agent never reports a state it did not fully apply
offline quiet for three poll intervals a reboot or a blip; often resolves itself
lost quiet for 48 hours stopped, uninstalled, firewalled, or the machine is gone

Each host reports the hash of the configuration it finished applying, and the server compares that against what it would serve the host now. Because an agent will not report a state it did not reach, a failed apply shows as out of sync rather than as success. Silence is tracked separately: three missed polls reads as offline, two days of silence as lost. Neither status revokes or deletes anything.

There is also a local config flag. NSClient++ reads local settings ahead of fleet-managed ones, so a locally set key shadows a pushed one. The agent reports that a host has local configuration โ€” never its content, which is where credentials tend to live โ€” so partly managed hosts are visible rather than reported as in sync when they are not.

Network and TLS

443 is the only port. The operator UI, the agents and certificate issuance share it, separated on the TLS handshake:

The client offers It gets
ALPN nsclient-fleet/1 pinned self-signed cert ยท client certificate required
anything else Let’s Encrypt cert ยท no client certificate requested
ALPN acme-tls/1 throwaway challenge cert โ€” handshake only

Agents pin a single certificate and do not consult the system trust store, so agent connectivity does not depend on certificate issuance being reachable, and on-prem and air-gapped installations behave the same as internet-facing ones. Browsers are never asked for a client certificate, so there is no certificate picker and no tenant information exposed to visitors.

The limitation this implies is worth planning for: anything terminating TLS between an agent and the server โ€” an inspecting proxy, a CDN that terminates TLS, most L7 load balancers โ€” breaks the agent connection. The TCP connection has to be passed through.

Security model

Identity comes from the certificate. The tenant is derived from the certificate’s issuer rather than from anything the certificate claims about itself, and the host id is a spiffe:// name cross-checked against that issuer. The server constructs the certificate; the request contributes a public key and nothing else. No request body carries a host id, token or identity.

Configuration and scripts are signed. SHA-256 for integrity and Ed25519 over the digest for authenticity, with the signing key kept separate from the CA, so a CA compromise does not allow bundles to be forged. Group membership is re-checked on every download, so a compromised host cannot fetch another group’s scripts.

An agent that cannot verify what it received does not apply it, does not fall back to an older unsigned state and does not go quiet: it keeps running the last known-good configuration and reports the failure.

One consequence is worth stating directly. By default the server can cause an agent to execute code, because deploying scripts is one of the things it is for โ€” the trust boundary is the management server, in the same way that the trust boundary for a scripting module is the script files. With require encrypted bundles enabled, that is no longer the case.

Known limitations

This is a 0.1.0 release, and the limitations are worth reading before planning around it.

  • It is new and will have bugs, the UI especially.
  • Templates will change, and the way bundles are bound to tags is likely to gain a richer language.
  • Agent upgrades are not implemented. They are planned for after the modern folder layout becomes the default.
  • Script dependencies are not handled, and may never be.
  • Fleet configuration on the agent, including the bundle key, is stored in clear text on every host. Encrypting it is on the roadmap.

Configuration written today is not at risk from any of this: the configuration format is owned by NSClient++ and is not changing.

Also planned: template variables for per-host values such as thresholds, more fleet-configuration management on top of the bundle-key and CA rotation that already exist, and adapters that feed discovered hosts into Nagios and other monitoring systems.

Running it alongside an existing setup

Fleet does not require migrating anything. The config-file and REST approaches remain supported, with no deadline attached. An enrolled agent does not have its nsclient.ini touched โ€” local settings are left in place and reported as local config โ€” and leaving a fleet is the same MSI properties pointed elsewhere.

A handful of non-critical hosts is enough to see the inventory, grouping and status behaviour with real data. Reports of what breaks are welcome on GitHub issues and in discussions.

Getting started

Download

Download NSClient Fleet from GitHub All downloads

// Michael Medin

0.21.0 NSClient Fleet launches: central management for your agents, plus checks that only read what you allow

0.21.0 is the agent release that goes with the first release of NSClient Fleet, a new, separate product: one server that holds the configuration for a whole estate of NSClient++ agents, hands each host the part that applies to it, and shows you what every host is actually running. Agents enroll themselves and report what they are, so the server builds the inventory instead of consuming one; groups select hosts by those reported tags, and bundles of configuration and scripts attach to groups. This release brings the agent side to feature parity with the server: encrypted bundles the server cannot read, a way to leave a fleet, and an nscp test prompt that survives a configuration push. Fleet is entirely optional; NSClient++ works exactly as it always has without it.

The other theme is reading less. check_logfile, check_files, check_wmi, check_pdh, check_registry_* and check_eventlog read whatever their argument names, with the agent’s privileges, which is a general read primitive wherever a caller may choose the argument. Each of them now has an access mode and an allow list, off by default. Two WEB roles complete the picture: restricted runs the checks you define but passes no arguments, and metrics scrapes and does nothing else. And the nscp test prompt got a round of usability work: aligned listings, a desc that shows defaults and what an alias runs, filter keyword lists, single-quoted paths and case-insensitive completion.

โœจ Highlights

  • ๐Ÿš€ NSClient Fleet is out. A server that enrolls your agents, builds the inventory from what they report, and delivers configuration and scripts per host as signed bundles, pulled by the agent over mTLS with no inbound port. Read the launch announcement and the Fleet documentation; this release is the agent side of it.
  • ๐Ÿ›ก๏ธ Access modes for the checks whose argument decides what is read. check_logfile, check_files, check_single_file, check_disk_write, check_wmi, check_pdh, check_registry_key, check_registry_value and check_eventlog each gained a mode setting and an allow list. The default, any, is exactly the previous behaviour; predefined limits a caller to the names you configured; allowed matches an allow list. Nothing changes on upgrade. (#1516)
  • ๐Ÿ”’ Two WEB roles that cannot be widened. restricted holds queries.execute.noargs, the REST equivalent of NRPE’s allow arguments = false: it runs the checks the agent defines and refuses any query-string parameter. metrics reads the two metrics endpoints and nothing else. The bundled monitoring role finally holds the grant the metrics endpoints actually check, so a monitoring user is no longer answered 403 there. (#1517, #1520)
  • ๐Ÿ” Encrypted fleet bundles. A bundle sealed in the fleet server’s browser is opened by the agent with a key you hand it out of band, at enrollment (nscp enroll --bundle-key, FLEET_BUNDLE_KEY on the MSI) or later. The key and the optional “sealed bundles only” posture live in the enrollment manifest, never in the settings store, so the server cannot plant or switch them. (#1521)
  • ๐Ÿšช nscp enroll --unenroll leaves the fleet. It removes the include, the identity and keys, and the fleet directory, and reports each step. (#1521)
  • ๐Ÿ’ฅ nscp test no longer crashes after a fleet configuration push. A settings reload replaced the object the prompt’s completion held a pointer to; the next completion refresh dereferenced freed memory. The crash file the agent writes now names the faulting module instead of printing a pointer. (#1521)
  • ๐Ÿ–ฅ๏ธ A better nscp test prompt. Padded tables instead of tabs in every listing, desc with parameter defaults, the bare-call command line and what an alias runs, a keywords verb listing a check’s filter keywords, alias as a shorter aliases, Tab completion that corrects load check to load Checkโ€ฆ, single quotes that take paths literally, exec that passes --options to a module, and a settings dump that lists only what is configured with passwords masked. (#1522)
  • ๐Ÿ”Ž Three follow-up fixes to the access gates from review, including a path written with the other separator walking out of an allowed directory, a link the path resolver skipped, and a reload window during which every gate stood open. Ship the release with those in, not the first cut.
  • ๐Ÿ“ฆ Packaging. Each Windows install no longer strands an 8 MB copy of nscp.exe under %WINDIR%\Installer, the WinGet manifests carry the metadata the upstream validator wants again, and the Debian source package drops a Unicode-licensed file so it passes Lintian. (#1518)

๐Ÿ” Detailed changes

๐Ÿš€ NSClient Fleet

Until now a large estate of agents was configured either by pointing each one at an ini file on a web server or by pushing settings over the REST API from whatever orchestration you already run. Both still work. Neither keeps track of the estate: the only inventory is the one you maintain by hand.

Fleet turns that around. A host enrolls with a one-time token, a few MSI properties or one nscp enroll command, and appears in the inventory with its OS, version, drives and detected roles as tags the agent reported. Groups select hosts by tag, bundles of INI fragments and scripts attach to groups, and each host pulls the bundles that apply to it, verifies their signatures, renders them into a fleet.ini its own nsclient.ini includes, and reports back what it applied. The server never pushes and never needs a port opened on a monitored host. Status is derived from what hosts report: in sync, out of sync, offline, lost, and whether local settings outrank what the server sends. It is one static binary and a SQLite file, on Linux or Windows or as a container, in its own repository.

To learn more, start with the launch announcement, then the Fleet documentation for running it in Docker, installing it on Linux or Windows, and the deployment reference. The agent-side walkthrough is Central management with NSClient Fleet. What this release adds on the agent side is below.

๐Ÿ›ก๏ธ Restricting what a check may read

The checks in the table take an argument that decides what data is read, and the agent reads it with its own privileges. Where callers choose the argument, NRPE with allow arguments = true or a REST user not on the restricted role, an unrestricted file= is a general file-read primitive. Each module gained a mode setting and an allow list:

Check Section Mode setting Allow list
check_logfile [/settings/logfile] file access allowed files
check_wmi [/settings/wmi] query access allowed classes, allowed namespaces
check_pdh [/settings/system/windows] counter access allowed counters
check_files, check_single_file, check_disk_write [/settings/disk] file access allowed files
check_registry_key, check_registry_value [/settings/system/windows] registry access allowed registry keys
check_eventlog [/settings/eventlog] log access allowed logs

The modes are any (the default, and what every earlier release did), predefined (the secure option: only names you configured in the module’s own sections, such as [/settings/logfile/files] or the counters already in [/settings/system/windows/counters]) and allowed (only what matches the list; experimental, since it has to parse what the caller sent). Registry and event-log entries are hierarchical and match whole name segments. Configured names resolve in every mode, so you can name your checks first and tighten the mode afterwards.

Once a mode is set, some arguments tighten with it: a check_wmi namespace= may no longer leave root\cimv2 unless allowed namespaces says so and its target= must name a configured target, check_registry_* refuses computer=, and check_eventlog’s default channels go through the gate like any other. Review of the gates before release closed a path written with the other separator walking out of an allowed directory, a symbolic link the path resolver did not follow, a NUL byte, a remote host in the path, and a window during a settings reload in which every gate stood open; allowed mode judges a WMI query by its class rather than its text. Alongside this, check_files on Windows no longer follows file symbolic links, as the Linux scanner never did, and * and ? in a path allow list no longer cross a directory separator (C:/logs/**.log for the subtree). See Restricting what a check may read.

๐Ÿ”’ WEBServer โ€” roles that cannot be widened

restricted holds queries.execute.noargs instead of queries.execute: the caller may run the checks the agent defines, and a request carrying any query-string parameter is refused with 403 Arguments are not allowed for this user. Neither grant implies the other. Give such a caller the checks that need arguments as aliases, so the arguments live in your configuration:

[/settings/WEB/server/users/monitor]
role = restricted

[/settings/check helpers/alias]
check_root_disk = check_drivesize drive=/ warning=free<10% critical=free<5%

Every query parameter counts, including a session token passed the legacy way as ?TOKEN=, so a restricted client authenticates with a header.

metrics is for a Prometheus scraper: metrics.list and openmetrics.list, no queries.execute. The bundled monitoring role granted metrics.get, a privilege nothing checks, so a monitoring user got 403 on both metrics endpoints; it now grants the two real ones. Roles already written to nsclient.ini are never rewritten, so an existing monitoring line keeps its inert grant until you update it or assign metrics instead.

๐Ÿ” Fleet โ€” encrypted bundles and unenrolling

A bundle the operator seals in the fleet server’s browser (format: enc-v1) used to be refused by the agent as an unreadable archive. The agent now opens it. The envelope is AES-256-GCM with the bundle’s name and version bound in as additional data, so a server that re-labels an old sealed bundle gets a refusal; the published checksum and signature cover the envelope, so download verification is unchanged and decryption is a step after it. The plaintext exists on disk only while it is unpacked; the cache keeps the envelope.

The key reaches the host out of band, never from the server:

Where How
At enrollment nscp enroll --bundle-key <key> (repeatable while rotating), or FLEET_BUNDLE_KEY=<key> on the MSI
On an enrolled host nscp enroll --update-bundle-keys --bundle-key <key>, or re-run the MSI with only FLEET_BUNDLE_KEY
Sealed bundles only nscp enroll --require-encrypted-bundles, or FLEET_REQUIRE_ENCRYPTED_BUNDLES=1

Both the keys and the requirement are stored in the enrollment manifest beside the host’s private key, not in nsclient.ini: the fleet-managed configuration is an include of the settings store, so anything kept there could be planted by the very server the bundles are sealed against. A bundle sealed with a key the host lacks is refused and the state report names the missing key’s fingerprint, which is what the server shows on its key page.

nscp enroll --unenroll removes the [/includes] fleet entry, the manifest and the fleet directory, in that order, and says what it removed; a service restart stops the sync. It is a local act, so remove the host on the server as well. Enrollment also resolves the manifest path from [/settings/fleet] state file on every path now; a host that sets that key used to enroll into a file the service never read. See Central management with NSClient Fleet, new in this release as a guide. (#1519, #1521)

๐Ÿ–ฅ๏ธ The nscp test prompt

A fleet configuration push, or any settings reload, killed the prompt: the reload re-entered the module and replaced the client object the completion hooks held a raw pointer to. Fixed, and the crash record the agent writes now names the faulting module (it printed a pointer). On top of that:

Verb Now
queries, aliases, list, plugins Padded tables, one line per entry; list really lists both kinds
desc <query> Parameter defaults, the command as it runs bare (show-default), and for an alias the command it runs plus that command’s parameters
keywords <query> The filter keywords and functions of a check with their descriptions, from the running agent
alias Same as aliases
settings Only what the configuration sets, with passwords masked
exec <module> --opt Options reach the module as nscp <module> --opt sends them, instead of --opt being taken for the command
load check<Tab> Completes and corrects the case: CheckDisk, CheckSystem, โ€ฆ
'C:\Program Files\x' Single quotes take their content literally; "..." keeps its backslash escapes, and filter=core='total' typed bare still reaches the check as written

A PDH enumeration race seen through exec CheckSystem --list inside the prompt is fixed as well: when the counter list grows between the sizing call and the fetch, the agent grows the buffer and retries instead of reporting PDH_MORE_DATA as a failure. (#1522)

๐Ÿ› Bug fixes

  • check_logfile files= works: the comma-separated form was parsed before the check read its arguments and had been ignored since it was added; it is one list with file= now, and every name goes through file access.
  • check_installed_software on Debian and Ubuntu takes install dates from dpkg-query (db-fsys:Last-Modified, dpkg 1.19.3 or later) instead of dpkg’s internal database, and returns UNKNOWN when a package’s file list cannot be read for a reason other than a missing file. (#1485)
  • The nscp test fallback on Linux appended a tab and ... to every line of multi-line output; it now prints the record as written, as Windows did.
  • Enrolling with a bundle key that has = padding in the middle is refused with a message that says so, and the MSI names the property that is actually missing when only FLEET_REQUIRE_ENCRYPTED_BUNDLES is given on an unenrolled host.

๐Ÿ“ฆ Packaging

  • The MSI’s Add/Remove Programs icon was nscp.exe itself, so Windows Installer extracted an isolated 8 MB copy of the agent into %WINDIR%\Installer on every install. It is a real icon now.
  • The WinGet manifests declare the VC runtime dependency, the real licence, the published moniker, ReleaseNotes, Documentations, locale, scope and installer switches again, date a manually re-published manifest by its release rather than the day the workflow ran, and keep the useful part of the release notes instead of cutting a table in half.
  • The vendored replxx used by the prompt no longer carries Unicode, Inc.’s ConvertUTF, whose licence is not DFSG-free; the Debian source package passes Lintian again. (#1518)

โš ๏ธ Upgrade notes

  • Nothing changes by itself. Every access mode defaults to any, no user is assigned to the new WEB roles, and roles already written to nsclient.ini are not rewritten. Set a mode, or update a monitoring role line, only when you want the new behaviour.
  • check_files on Windows skips file symbolic links. If you relied on them being counted, point the check at the link targets. Path allow-list wildcards no longer cross directory separators; use ** for a subtree.
  • check_logfile with both file= and files= reads the union from this release on. Nothing to do unless you relied on files= being dropped.
  • check_installed_software on a dpkg older than 1.19.3 leaves install_date unset, so an expression on it no longer matches there.
  • Encrypted fleet bundles need the key on every host before the server starts serving sealed bundles; a host without it refuses them and names the missing key’s fingerprint in its state report. There is no key escrow on the server.

Security notices for this release: Access modes for the checks whose argument decides what is read, WEB: a metrics role, and a corrected metrics grant on the monitoring role, Fleet: encrypted bundles are opened by the agent, not the server and Sensitive settings are redacted in the nscp test settings dump. The full list of behaviour changes is on the upgrading page.

Download

You can download the new version from GitHub

// Michael Medin

0.20.0 .NET plugins return, the WEB server keeps passive results, and more security fixes

0.20.0 brings back .NET plugins โ€” on Linux as well as Windows this time โ€” and lets the WEB server hold on to passive check results so a monitoring server can collect a host’s scheduled checks in a single request instead of being pushed to. A security review of NRPE and the TLS layer underneath it fixed a handful of findings; the one to act on is that TLS certificates the agent generated itself were readable by every local user, and a generated CA handed its private key to every client it was distributed to.

The release also fixes some seventy crash and hang bugs found by scanning the whole code base after the check_service crash in 0.19.0, closes two gaps in the credential guard introduced in 0.19.0, and makes stopping the service prompt again on Windows hosts where it could hang for minutes.

โœจ Highlights

  • ๐Ÿ”Œ .NET plugins are back, on Windows and Linux. The DotnetPlugins module hosts an installed .NET runtime (8.0 or newer), so plugins written in C# or F# load on both platforms. Nothing changes on a default install: the module stays off until you enable it, and no runtime is bundled. Plugins built for the old .NET Framework API need a rebuild against the new NSCP.Core.dll; the interfaces are the same. (#1481)
  • ๐Ÿ“ค The WEB server can keep passive results and serve them over REST. Turn the result cache on and everything submitted to its channel โ€” typically scheduled checks โ€” waits in the agent until something polls GET /api/v2/results. The bundled check_nsclient 1.1.0 adds results feed, which collects that cache and hands every entry to Nagios as a passive result: one active check per host instead of one per service, and no NSCA or NRDP receiver needed. A new scenario walks through the Nagios Core setup. (#1498)
  • ๐Ÿ”’ Generated TLS keys are private now; check yours. Certificates the agent generated itself โ€” including the one a default NRPE start creates โ€” were written world-readable, and a generated CA wrote its private key into the ca.pem meant for clients. New files are created correctly; existing ones are left alone, so see the upgrade notes. (#1496)
  • ๐Ÿ›ก๏ธ NRPE and TLS hardened. A TLS handshake can no longer be left open forever by a permitted host, a short NRPE packet can no longer smuggle unverified bytes into the command, the NRPE client tells you when it is not authenticating the server, and the Logjam-broken 512-bit DH parameter file is gone. (#1496)
  • ๐Ÿ› Seventy crash and hang fixes across the agent. Three could be triggered from outside: a malformed HTTP response from any server the agent talks to could hang it, an empty console command over REST could crash it, and filter_perf sort=normal could crash on the Nagios U marker. The rest were bugs on the default paths of common Windows checks, races when a module is reloaded or unloaded, and thresholds that silently overflowed. (#1505)
  • ๐Ÿ” Two ways past the credential guard closed. A token or password written inside a target’s address is now protected the same way as one in its own key, and a target without an address no longer adopts the caller’s host as its own.
  • โฑ๏ธ Stopping the service is prompt again. On Windows, stopping shortly after start could take minutes while the Windows Update check or a WMI query finished; those are now abandoned on shutdown. (#1504)
  • ๐Ÿง Builds with GCC 16 for anyone building from source on a current Debian or Fedora. (#1479)

๐Ÿ” Detailed changes

๐Ÿ”Œ DotnetPlugins โ€” .NET plugins on Windows and Linux

The old module only ever worked on Windows and had been out of the build for years. The new one finds an installed .NET runtime โ€” DOTNET_ROOT, the registered install location or the platform’s default folders, or a path you give in runtime path โ€” and hosts your plugins through it. Everything a plugin could do before still works: commands, submission channels, command-line exec and log messages all reach the managed side.

[/modules]
DotnetPlugins = enabled

[/settings/dotnet/plugins]
MyPlugin = MyPlugin.dll

Assemblies are looked up in plugin path (modules/dotnet by default, where NSCP.Core.dll lives too). On Windows the installer has a “.NET plugin support” feature again, selected by default; the Linux packages ship the same files. See Extending with .NET and the DotnetPlugins reference.

๐Ÿ“ค WEBServer โ€” a passive result cache served over REST

Passive monitoring normally means the agent pushes results to the monitoring server. That does not work when the server has no NSCA or NRDP receiver, or when the agent cannot reach it. The WEB server can now turn it around: with the cache enabled it listens on a submission channel (WEB by default), keeps whatever arrives there โ€” scheduled checks from Scheduler, check_and_forward from CheckHelpers, anything that submits to a channel โ€” and hands it over when polled.

Setting (/settings/WEB/server/results) Default Meaning
enabled false Turn the cache on (takes effect on restart)
channel WEB The submission channel to listen on
primary index ${host}/${alias-or-command} What makes two results “the same check”
mode last Which of two results for a check to keep: the newest (last) or the most severe (worst)
clear on poll true Polling empties the cache, so worst means “worst since the last poll”
max entries 1000 How many checks to keep before the oldest is dropped
max age none Drop results older than this

The cache is exposed as GET /api/v2/results, GET /api/v2/results/{key}, DELETE /api/v2/results and DELETE /api/v2/results/{key}. They need the new results.list, results.get and results.delete privileges, which only the full role has, so grant them to the account that polls:

nscp web add-role --role poller --grant results.list,results.get,login.get

check_nsclient 1.1.0, bundled with the Windows and Linux packages, adds the results list, results show, results delete, results clear and results feed commands; feed is the one to schedule from Nagios. The REST results page describes the API and the Polled Passive Checks (Nagios Core) scenario the complete setup.

๐Ÿ”’ NRPE and TLS โ€” what the review found

None of the findings lets anyone past allowed hosts run code on the agent.

  • Generated private keys were readable by every local user, and a generated CA put its private key into the ca.pem you distribute to clients โ€” anyone holding that file could mint certificates the server accepts with verify mode = peer-cert. New keys are created readable only by the agent, and a generated CA keeps its key in a separate ca-key.pem.
  • A TLS handshake could be left open forever. A permitted host could open connections, send nothing, and hold them indefinitely. The listener’s timeout now covers the handshake as well.
  • The NRPE client did not authenticate the server. With ssl = true and the default verify mode = none the link is encrypted but anyone on the path could impersonate the server. The default stays, but the client now logs one error per target so the choice is visible, and generated certificates name the machine rather than only localhost so verification can actually be turned on.
  • A short NRPE v3/v4 packet could carry extra bytes into the command that the checksum never covered.
  • The 512-bit DH parameter file is no longer shipped. Nothing used it by default, but anyone who copied it into dh was running a Logjam-broken key exchange.
  • Smaller fixes: tls version accepts every spelling the documentation lists (tlsv1.3+, 1.0+, sslv3+, any, โ€ฆ used to fail with “Invalid tls version”, which for an NRPE listener showed up as “listener failed to start”), and several NRPE wire-format bugs are fixed, including version 4 packets not being recognised by the server.

๐Ÿ” Client modules โ€” the credential guard covers the address too

0.19.0 stopped a caller from redirecting a target that carries a credential to a host of their choosing. It missed two cases: a secret written inside the address itself (?token=SECRET in the URL, or user:password@host), and a target that had a credential but no address, which took the caller’s host as its own. Both are closed. If you relied on either, the options are the same as in 0.19.0: pass the credential with the request, configure one target per destination and pick it with target=, or set allow host override = true.

๐Ÿ›ก๏ธ Crash and hang fixes across the agent

After the check_service crash in 0.19.0 (#1499) the whole code base was checked for the same kind of mistake, and roughly seventy were fixed. Most were on paths ordinary checks take every day: reading event log records, WMI results and scheduled tasks, taking process snapshots, check_cpu and check_pagefile. Others showed up when a module was reloaded or unloaded while a check was running, or when a threshold or unit suffix was larger than the agent could represent. As a side effect, a few inputs that used to do something odd are now rejected outright:

Input Now
check_cpu time=0 An error: the window must be at least one second
A threshold or unit that overflows (used > 1.0e30T, time=5000000w) An error instead of a wrapped-around value
A module that failed to load Dropped from the module list; no longer answers exec
NSClientServer (check_nt) or CheckMKServer on a settings reload Restart their listener so a changed port or password takes effect; open connections drop
A Python script unloading PythonScript from inside Refused

โฑ๏ธ CheckSystem and CheckDisk โ€” shutdown no longer waits on Windows Update or WMI

Stopping the service shortly after it started could take minutes on Windows (#1504). The first Windows Update check the agent runs goes online to Windows Update or WSUS and cannot be interrupted, and the service waited for it. The WMI queries behind the network, temperature, CPU frequency, battery and disk I/O collectors could hold things up the same way while the WMI performance service restarts. All of them are now abandoned when the service stops.

๐Ÿง Building from source

The tree builds with GCC 16 (Debian unstable, Fedora rawhide), which defaults to C++20 (#1479).

๐Ÿ“š Documentation

โš ๏ธ Upgrade notes

  • ๐Ÿ”’ Check the permissions of certificates the agent generated for you. The upgrade does not touch existing files: run chmod 600 /etc/nscp/security/certificate.pem (or restrict the file to the service account on Windows). If you handed out a generated ca.pem, regenerate that CA and re-issue client certificates โ€” the old file contains the CA’s private key.
  • ๐Ÿ”’ If dh names nrpe_dh_512.pem, change it before upgrading to ${nrpe-dh}/nrpe_dh_2048.pem; the file is no longer shipped and the listener will otherwise fail to start. An existing copy on disk is left where it is.
  • ๐Ÿ”’ The NRPE client logs an error for every target it does not authenticate. The default (verify mode = none) is unchanged. Set verify mode = peer-cert with ca pointing at the issuer to authenticate the server and silence it. Regenerate an existing generated certificate to get one that can be verified.
  • ๐Ÿ”’ Slow clients may be dropped during the TLS handshake. The handshake now has to finish within the listener’s timeout (30 s by default). Raise timeout if a client on a slow link stops connecting.
  • ๐Ÿ”’ A credential inside a target’s address now blocks host= overrides, as one in password or token already did. Use target=, pass the credential with the request, or set allow host override = true.
  • ๐Ÿ”’ Some inputs that used to misbehave are rejected: see the table above. Nothing to do unless you relied on one of them.
  • ๐Ÿ”Œ DotnetPlugins is available again but not loaded unless you add DotnetPlugins = enabled under [/modules], and it needs a .NET runtime (8.0 or newer) installed on the host. Plugins built against the old .NET Framework NSCP.Core.dll must be rebuilt against the new one.
  • ๐Ÿ“ค The result cache is off by default. Enabling it needs a service restart, and the account that polls needs the results.* privileges. The bundled check_nsclient is now 1.1.0; existing invocations are unaffected.
  • ๐Ÿ”ง Every documented tls version spelling works now. Nothing to do; the default tlsv1.2+ was never affected.

Full detail on the security items lives in Security notices; the operator actions are mirrored on Upgrading.

Download

You can download the new version from GitHub

// Michael Medin

0.19.0 A real nscp test prompt, a crash fix for service filters

0.19.0 gives the interactive console a proper prompt โ€” line editing, history, tab completion and highlighting โ€” and fixes a heap-corruption crash that took the whole agent down whenever a check_service filter matched nothing (#1499). A whole-codebase security review closed three findings: a client module’s configured credential could be sent to a caller-chosen host, REST script uploads were staged at a predictable path, and a junction defeated the modern-layout lockdown of %ProgramData%\NSClient++. The collectd client was reworked end to end โ€” host names resolve, timeout and retries are honoured, failed sends are reported, datagrams are sized correctly and a multicast target no longer fans out over every local interface.

Alongside that, the WEB server’s authentication limiter escalates against rapid-fire guessing, NRDP warns about an unverified TLS link, Icinga honours a base path in the target address, every documented query now has prose and captured samples, and the Upgrading and Security notices pages are assembled from one file per note with a module/version/action filter.

โœจ Highlights

  • ๐Ÿ–ฅ๏ธ nscp test is a real prompt. On a terminal you get line editing, persistent per-user history, position-aware tab completion against the command registry, hints, and highlighting that turns an unknown query or module name red before you press enter. Log messages redraw around the line you are typing instead of landing in the middle of it. Piping commands in now works on Windows, and an exhausted stdin no longer spins a core at 100%. (#1488)
  • ๐Ÿ›ก๏ธ A check_service filter that matched no service no longer kills the agent. check_service "filter=name = 'nosuchservice'" โ€” or a filter that merely missed on case โ€” terminated nscp with exception code 0xC0000374 and no result. Both check_service keywords and check_logfile’s column() now answer the documented empty-result contract, and every optional read in the tree goes through .value() so a future miss is a reported error, not a write to freed memory. (#1499)
  • ๐Ÿ” Client credentials stay with their target. host=, port= and address= moved a submission’s destination while the target’s configured password or token came along, so any holder of queries.execute could have the agent post the NRDP token, the Icinga login, the SMTP login or the NSCA password to a host of their choosing. That combination is refused now; target= also works for queries, and allow host override = true restores the old behaviour per target. (#1492)
  • ๐Ÿ”’ Two more review findings closed. PUT /api/v2/scripts/โ€ฆ staged the upload at ${temp}/<name>, where a local user could plant a file of the same name and have it imported as a command; it is staged in a randomly named, owner-only file now. On the opt-in modern layout, a pre-created junction at %ProgramData%\NSClient++ had the lockdown secure the junction’s target; reparse points are refused and the installer, the migration and service start all fail on one. (#1492)
  • ๐Ÿ“ก The collectd client works the way its settings say. A target named by host name threw on every metrics cycle; timeout and retries were read and ignored; a failed send looked exactly like a delivered one; a value list of a few hundred entries overflowed the 1452-byte datagram the receiver reads; and a multicast target sent a copy through every local interface, DMZ and guest NICs included. All fixed, with a new per-target multicast interface setting (auto, the default, all, or a list of local addresses). (#1494)
  • ๐Ÿ”’ Failed WEB logins back off exponentially. The fixed 60-second block after ten failures let an attacker resume at a steady rate forever โ€” about 14 000 guesses a day per address. Each further block now doubles up to an hour, but only for a run of failures that burned the whole budget at machine speed, so a client retrying a stale password behind NAT cannot lock out everyone sharing its address. (#1493)
  • ๐Ÿ”ง The console log is no longer held in a 64 KB buffer, so nscp test shows log lines as they happen instead of when you press a key, and a redirected or supervised console streams. --no-stderr and the oneline format finally take effect. (#1488)
  • ๐Ÿ“š Every documented query has a description and samples, captured against a running agent, with the errors the capture turned up corrected in the text. The Upgrading and Security notices pages are now built from one file per note, with a filter for the version you come from, the modules you run and whether a note needs action. (#1480, #1482)

๐Ÿ” Detailed changes

๐Ÿ–ฅ๏ธ CommandClient โ€” nscp test gets a real prompt

The interactive console was a poll loop around std::getline: no line editing, no history, no colour. When both stdin and stdout are a terminal it now runs on replxx, vendored under libs/replxx/ (byte-identical to upstream so provenance can be diffed; no network needed at build time).

History Persistent, per user, saved after every command (nscp test is routinely killed). %APPDATA%\NSClient++\console-history.txt on Windows, $XDG_STATE_HOME/nscp/console-history or ~/.nscp_history elsewhere, created 0600 on POSIX.
Completion Position-aware: built-in verbs and registered queries in command position, query names after desc, the query’s own parameter names as name= once you are typing arguments. load/enable offer the modules that are not yet loaded or enabled, unload/disable the ones that are.
Highlighting A query or module name that does not resolve turns red before you press enter.
Hints The command’s one-line description, greyed after the cursor.
Log The agent logs from a background thread the whole time the prompt is up; messages are drawn above the prompt and the half-typed line redrawn underneath. Multi-line results keep their line breaks.

Commands typed at a prompt can carry credentials, so a new [/settings/cli] section controls what is kept: history size = 0 turns persistence off, history file relocates it, color disables colour.

The first load <tab> of a session pauses while the core scans the module directory; it is done once per process. help is now generated from the same vocabulary as the prompt, so it lists all sixteen built-in verbs instead of the eight it had drifted to, and an empty Performance data: line is no longer printed after every result that has none. See Test mode.

With stdin not a terminal nothing changes โ€” no prompt, no history, no colour โ€” except three fixes: piping commands in now works on Windows (the readiness check used a console-only API and silently ignored a file or pipe), an exhausted stdin parks the loop instead of spinning at 100% CPU on POSIX, and end of input is no longer treated as a reason to exit, which is how the agent is normally started under a supervisor.

To make this possible a module can now take the console over: the new NSAPISetLogOption core API accepts the same strings as the --log switch, and the prompt calls set_log_option("no-console") while it owns the terminal. Fixing the one-way console flag exposed that oneline and no-std-err were being forwarded to the log level parser, rejected with Invalid log level: no-std-err, and never applied. Both reach the log driver now.

๐Ÿ”ง Core โ€” the console log is flushed

The console log backend installed a 64 KB buffer on standard output and nothing ever emptied it. MSVC’s stream honours that buffer, so on Windows log output sat there until something else flushed the stream โ€” in nscp test that was reading the next line of input, which is why the log appeared to catch up only when you pressed a key. A redirected console (nscp test > log.txt, a container, a supervisor) looked mute until 64 KB had built up or the process exited. Every message is flushed as it is written now.

๐Ÿ›ก๏ธ Filters โ€” an empty filter result no longer corrupts the heap

When nothing matches a filter, the framework re-evaluates the warning and critical expressions with no object bound to the evaluation context, so that an expression which also reads the summary (โ€ฆ or count = 0) still reaches a verdict. check_service defaults to not state_is_perfect() and not state_is_ok(), and both read the service straight off the context without checking one was there. That dereferenced an empty optional, resurrecting a destroyed shared_ptr control block out of the vacated storage; the copy taken of it wrote to freed heap memory, and the process died somewhere unrelated with 0xC0000374 and no usable stack. debug=true masked it, because with debug on the context keeps a copy of every object and the stray write lands on live memory. check_logfile’s column() keyword had the same unguarded access.

Both keywords now report an unresolved value when no object is bound, as the built-in keywords already did, and the check returns UNKNOWN: No services found. The accessor underneath throws a filter error instead of reading the vacated storage. Any host past allowed hosts could trigger this over NRPE with allow arguments = true, and any authenticated REST client could; the Unix implementation already had the guard and was never affected. (#1499)

As a follow-up, all 214 optional dereferences across 67 files were converted to .value(), including the ones sitting under an if (opt) guard: the guard is what a later edit moves or deletes, and uniformity is what makes the rule checkable. .value() throws bad_optional_access, which the catch around every filter evaluation turns into a reported error on the check.

๐Ÿ” Client modules โ€” credentials pinned to their target

The shared client parser loads the module’s default target โ€” credential included โ€” and then applies the request’s arguments on top. host=, port= and address= moved the destination while the credential stayed, so

GET /api/v1/queries/submit_nrdp/commands/execute?address=http://attacker.example/nrdp/&command=x&result=0&message=x

had the agent post the configured NRDP token to the attacker. Both seeded REST roles carry queries.execute and the permission policy is off by default, so a checks-only REST user was enough; over NRPE it needed allow arguments = true. Affected are the modules whose targets carry a credential: NSCA, NSCA-NG, NRDP, Icinga, SMTP and NSCP.

A request that moves the destination away from the target’s configured address is now refused when the credential that would travel is the target’s own. The guard decides on two facts โ€” the resolved destination differs from the one the target configured, and at least one credential still in the container is the target’s rather than the request’s โ€” so a target with no credential, a request that supplies its own password=/token=, and a request that does not move the destination are all unaffected, and a destination moved through a header host entry is caught too. allow host override = true on a target restores the old behaviour explicitly.

target= now selects a configured target on the query path as well. It was only ever applied when a command ran as an exec; as a query โ€” which is what a REST or NRPE caller gets for check_* and submit_* โ€” it was accepted and silently ignored, so the one remedy the refusal recommends did not work where the refusal is most likely to be met.

๐Ÿ”’ WEBServer โ€” script uploads are staged privately

PUT /api/v2/scripts/โ€ฆ (admin only) wrote the body to ${temp}/<name> โ€” /tmp, or C:\Windows\Temp for a SYSTEM service โ€” with an unchecked truncating write, then imported it as a command. A local user who created that file first won a race against the copy, or won outright where the service’s overwrite was refused and the failure ignored, and the planted content then ran as the service account. Stock DEB/RPM installs were not exploitable for code execution (the service runs as nsclient and the script root is root-owned). Uploads now go to a randomly named file, created exclusively and owner-only, never through a symlink, with every write checked and the file removed once consumed; a staging failure is reported as HTTP 500 instead of importing whatever was on disk.

๐Ÿ”’ Windows modern layout โ€” the shared folder must be a real directory

The opt-in, experimental LAYOUT=modern install keeps nsclient.ini, the fleet private key and the TLS material in %ProgramData%\NSClient++ and locks the folder down by taking ownership and replacing its DACL. Every step was path-based, and a standard user can create a junction under that name before the installer first runs: the owner and DACL were applied to the junction’s target while the link stayed theirs to swap for a real folder with a crafted nsclient.ini, which the next service start loaded as SYSTEM.

Ownership and the DACL are now applied through a handle opened on the entry itself (FILE_FLAG_OPEN_REPARSE_POINT, one open per operation asking only for the rights that operation needs), and anything that is not a plain directory is refused โ€” by the installer, by nscp settings --migrate-layout modern, and at service start. Legacy installs are untouched.

๐Ÿ“ก CollectdClient โ€” the sender reworked

Problem Fix
A target address written as a host name threw on every metrics cycle (make_address() accepts IP literals only), so metrics silently never left. Addresses are resolved; an unresolvable target is reported by name. The first endpoint the resolver returns is used.
timeout and retries were read into the connection and never used; the send was asynchronous and discarded its error code, so an unreachable target, a full socket buffer or an oversized datagram looked exactly like a delivered packet. The send is synchronous and checked. A locally failed send is retried up to retries times (default 3, 20 ms apart); the whole send, name resolution included, runs under timeout (default 30 s, 0 for no limit); failures are logged once per distinct message. A datagram the receiver already has is never sent twice.
Nothing bounded a values part, so a value list of a few hundred entries overflowed the 1452-byte datagram the collectd network plugin reads, and past ~3600 entries the part length wrapped and put a malformed part on the wire. One overlong host name could consume the whole packet. Each metric is costed against what the packet has left and flushed first when it does not fit; identifiers are clamped to 127 bytes (what the receiver stores); values that cannot fit a datagram of their own are counted and the number dropped is logged. Packets fill to the real limit again instead of stopping at roughly half.
A multicast target (no address configured โ†’ 239.192.74.66:25826) sent a copy of every datagram through every local interface of the matching family โ€” unauthenticated cleartext host name, CPU, memory, uptime and process counts on every attached segment. On a Debian-style host whose name maps to 127.0.1.1 the enumeration yielded loopback only, so nothing left the machine at all. New per-target multicast interface: auto (default) sends one copy through the interface the routing table picks; all restores the fan-out; a comma-separated list of local IP addresses sends through exactly those, with an unusable entry reported and skipped and a wholly unusable list sending nothing rather than falling back to the default route.
sent counted datagram ร— socket while failed counted payloads, so the “not sent” count underflowed to about 1.8 ร— 10ยนโน on a multi-interface target. A datagram is one unit of work whatever the interface count; sent + failed accounts for every payload.

The UDP delivery half moved out of the module into net/collectd/, where it is unit-tested against a real loopback socket, and the integration suite gained a target named localhost.

๐Ÿ”’ WEBServer โ€” escalating block on repeated authentication failures

The per-IP limiter blocked a client for a fixed auth rate limit block seconds (default 60) after auth rate limit max failures (default 10) consecutive failures and then reset its counter โ€” roughly 14 000 guesses a day per source address, indefinitely, against Basic auth, the password header and the legacy ?password= form check_nscp_api uses. Each consecutive block from the same IP now doubles the wait, up to an hour; a configured block already longer than that is used as configured. The escalation resets on a successful authentication or after an hour of quiet.

Only a run of failures that burned the whole budget faster than one attempt every two seconds escalates. The limiter keys on the socket peer, so behind NAT or a reverse proxy every client shares one address, and a single monitoring client retrying a stale password on a schedule must not be able to ratchet that address up to the ceiling. Bearer / ?TOKEN= session tokens are not metered: they are 256-bit random values, and counting an expired one against the limit would let a client with a stale session lock its own address out. This is defence in depth on top of PBKDF2 and the uniform 403; IP rotation remains out of scope.

An https submission whose verify mode carries no peer-verifying token sends the token โ€” a shared secret โ€” to whichever server answers. The module now logs that, naming the endpoint, once per target for the life of the process (the Icinga client already did; a first cut logged on every submission, which at a 60-second schedule is 1 440 lines a day). The connection itself is unchanged. The verify mode help text is corrected in the same pass: it recommended none for self-signed certificates and listed client-once, workarounds and single, which the client-side parser rejects. Use peer-cert with ca pointing at the certificate instead.

๐Ÿ”ง IcingaClient โ€” a base path in the target address is honoured

The path of a target address (https://proxy.example.com/icinga/) was parsed into a field nothing read, so every call went to /v1/โ€ฆ on the host and an Icinga 2 master published under a reverse-proxy subpath could not be reached. The prefix is normalised once (a doubled leading slash collapses to one, no trailing slash) and prepended to every API path, for the submission and the ensure-objects calls alike. Addresses without a path are unchanged.

๐Ÿ“š Documentation

  • **Upgrading and Security notices can be filtered. The reader picks the version they come from, ticks the modules they run, and can restrict to security-relevant or action-needing notes; the selection is remembered and mirrored into the query string.
  • Test mode has a page โ€” keys, what completion offers, where history is kept and how to turn it off, non-interactive behaviour, and the stop-the-service dance on both platforms.
  • REUSE metadata matches the bundled headers: the asio and SimpleIni overrides carried copyright years from later releases than the copies vendored here. (#1486)

โš ๏ธ Upgrade notes

  • ๐Ÿ”’ A check_service filter that matched no service no longer kills the agent. Nothing to configure. If you worked around it with service=<exact name> and no filter=, service name patterns are usable again.
  • ๐Ÿ”’ A client target no longer lets a request send its configured credential to a destination the request names. A submit_*/check_* call that passes host=, port= or address= against a target with a password or token, without supplying the credential itself, now fails with an error naming the target. Pass the credential with the request, configure each server as its own target and select it with target= (which now works for queries too), or set allow host override = true on the target. Targets without a credential are unaffected.
  • ๐Ÿ”’ REST script uploads are staged in a private, randomly named file. No configuration change; a staging failure is now an HTTP 500.
  • ๐Ÿ”’ Modern layout (opt-in, experimental): the shared folder must be a real directory. A %ProgramData%\NSClient++ that is a junction or symbolic link is refused by the installer, by nscp settings --migrate-layout modern, and at service start. Relocate the folder with a [paths] override in boot.ini instead. Legacy installs are unaffected.
  • ๐Ÿ”’ A multicast collectd target now sends through one interface, not all of them. If you depend on a multicast target reaching several segments, set multicast interface = all on it, or list the local addresses to send through. Unicast targets ignore the setting.
  • ๐Ÿ”’ Repeated rapid-fire failed WEB logins from one IP are blocked for longer each time. Nothing to do on a default install. A probe that deliberately authenticates with bad credentials will be blocked for longer; auth rate limit max failures = 0 still disables the limiter for a test harness.
  • ๐Ÿ”’ An NRDP submission over an unverified https link now says so in the log, once per target. If it names a target you expected to be verified, set verify mode = peer (or peer-cert with a ca).
  • ๐Ÿ”ง nscp test writes a per-user history file. Commands typed at the prompt can carry credentials; set history size = 0 under [/settings/cli] to keep nothing on disk. With stdin not a terminal nothing changes, except that piping commands in now works on Windows and end of input no longer exits.
  • ๐Ÿ”ง The console log is no longer buffered, and --no-stderr and the oneline log format now take effect. Nothing to do unless you were working around either.
  • ๐Ÿ”ง target= now selects a configured target on the query path. A REST or NRPE query that passed target= and relied on reaching default anyway will now reach the target it named.
  • โฑ๏ธ collectd submissions honour timeout and retries and report failed sends. Nothing to do unless you set a large retries on a collectd target โ€” it now costs real time, bounded by timeout (default 30 s; timeout = 0 for no limit).
  • ๐Ÿ”ง An Icinga target address with a path prefix is now sent. If a target address carries a path that is not a subpath of the API, remove it.

Full detail on the security items lives in Security notices; the operator actions are mirrored on Upgrading.

Download

You can download the new version from GitHub

// Michael Medin

0.18.1 Security hardening and bugfixes of monitoring clients

0.18.1 is a security and correctness release for the passive/outbound side of the agent. A review pass over the client modules โ€” Icinga, NRDP, NSCA, NSCA-NG, check_mk, Elastic, Graphite, syslog, and a second round on SMTP โ€” turned up the same three shapes over and over: configuration that parsed fine and was then thrown away, network operations that could never time out, and attacker-influenced text reaching another system’s log unscrubbed. The external-script launcher and the filter framework got the same treatment.

The most consequential single item is NSCA-NG cert mode, which applied its TLS configuration too late and therefore accepted any server certificate. Alongside the hardening, the Elastic module can finally talk to a current Elasticsearch, the web UI moves to MUI 9 / react-router 8 with its test suites wired into CI, and a coverage sweep adds unit tests to every source file that was under 50%.

โœจ Highlights

  • ๐Ÿ” NSCA-NG cert mode verifies the server again โ€” and presents your client certificate. The OpenSSL context was configured after the TLS stream was created from it, and SSL_new() copies the verify mode, certificate, cipher list and version bounds at creation time. So verify mode = peer-cert ran with verification off, any certificate was accepted, and the configured client certificate was never sent. The default PSK mode was never affected. (#1461)
  • ๐Ÿงพ Configuration that silently did nothing now applies. check_mk client targets discarded every TLS key (use ssl, certificate, verify mode, โ€ฆ) and connected in plaintext regardless; syslog targets never read severity, facility, tag_syntax or message_syntax; NSCAServer’s performance data = false was ignored; and ext-scr install --arguments=โ€ฆ wrote its lockdown to a key the module does not read. All four are fixed.
  • โฑ๏ธ A stalled endpoint can no longer wedge a submitting thread forever. Icinga, NRDP, Graphite and Elastic submissions all ran with no deadline; each is now bounded by the target’s timeout (default 30 s) as a single budget over resolve, connect, handshake and exchange. External scripts enforce their timeout by wall clock on both platforms. (#1464, #1465, #1466, #1467, #1468, #1453)
  • ๐Ÿ”Ž The Elastic module works against Elasticsearch 8 โ€” and verifies TLS. Verification was hardcoded off; it now defaults to peer with tls version, verify mode and ca settings, plus new user/password and api key authentication. The legacy _type parameter is no longer sent by default, and every document in a bulk request gets its own _id โ€” previously they shared one and overwrote each other. (#1453)
  • ๐Ÿ”’ tls version = 1.2+ means “1.2 or later” again. The + was stripped and the value mapped onto a version-pinned method, which pins the maximum too โ€” so the common default negotiated TLS 1.2 only and silently excluded TLS 1.3. Fixed in the shared stack: all HTTP-based clients, the NRPE/NSCA socket clients and servers, and check_tcp. (#1464)
  • ๐Ÿ“จ Syslog datagrams carry the RFC 3164 HOSTNAME field, so a conforming receiver stops promoting the tag to origin host โ€” which meant check output could choose which host a record was filed under. An unknown severity or facility now degrades to <13> (user.notice) instead of <0>, kernel.emergency. (#1470)
  • ๐Ÿงฑ Filter expressions are bounded at 1024 characters and 64 nesting levels. The parser and the AST evaluator both recurse with the shape of the input, so a long or deeply nested expression could exhaust the stack and crash the agent. Real filters sit an order of magnitude below the limits. (#1469)
  • ๐Ÿ–ฅ๏ธ The web UI moves to MUI 9, react-router 8 and TypeScript 6, with its 65 vitest unit tests and 18 Playwright integration tests now running in CI on every build. (#1459)

๐Ÿ” Detailed changes

๐Ÿ” NSCA-NG โ€” cert mode applies TLS configuration before the stream exists

use psk = false targets built the ssl::context, created the connection from it, and then set the verify mode, client certificate, cipher list and TLS version bounds. OpenSSL copies all of that out of the context when the stream is created, so none of it took effect: verify mode = peer-cert accepted a man-in-the-middle’s certificate, and a server asking for a client certificate never got one. Configuration is applied first now.

Two visible consequences: a cert-mode target whose server certificate does not chain to the configured ca (or does not match the host name) will now fail to connect โ€” that is the verification working โ€” and servers requiring a client certificate will start receiving it. The default PSK mode authenticates both ends through the pre-shared key and is unaffected.

๐Ÿงพ Settings that were read but never applied

Module Setting(s) What happened
CheckMKClient use ssl, certificate, certificate key, ca, allowed ciphers, verify mode, dh The target object never called register_all()/notify(), so the keys were parsed and thrown away โ€” the client connected in plaintext whatever the configuration said, and the keys were missing from the reference docs.
SyslogClient severity, facility, tag_syntax, message_syntax, ok-severity/warning-severity/critical-severity/unknown-severity Stored under keys the sender never read, so built-in defaults always won: a settings-defined target sent an empty tag and dropped the message text.
NSCAServer performance data = false Ignored; perfdata is stripped from forwarded submissions again, as documented.
CheckExternalScripts ext-scr install --arguments=โ€ฆ Wrote the lockdown to a path nothing reads, so it reported success while leaving arguments enabled. Re-run it after upgrading.
GraphiteClient timeout Looked up in the free-form option map, where the well-known timeout key is not stored โ€” the default 30 always won.

Values you configured โ€” possibly years ago, without effect โ€” now apply. Review those target sections for stale keys before upgrading.

โฑ๏ธ Operations that could never time out

Each of these ran with no deadline, so an endpoint that accepted the connection and then went silent held the submitting thread indefinitely, quietly stopping passive results until a service restart.

  • Icinga, NRDP, Graphite (both the submission and the recurring metrics flush) and Elastic are now bounded by the configured timeout (default 30 s; 10 s for one-shot nscp client submissions) as one budget covering name resolution, connect, TLS handshake and the exchange. NRDP also retries transport failures up to retry, each attempt on a fresh connection.
  • Graphite’s retry is gone. It never had any effect โ€” the module always made exactly one attempt โ€” and a retry loop would multiply the worst-case time a stalled endpoint can hold a thread. It is still registered centrally for all client modules, so it remains in the reference, but GraphiteClient does not act on it. Mirrors the SMTP retry change in 0.18.0.
  • External scripts. On Unix the single-string shell fallback ran through popen(), which hides the child PID, so timeout= was unenforced and a hung script wedged a worker thread per invocation. On Windows the read loop counted iterations rather than elapsed time, so a continuously chatty script escaped the timeout entirely and leaked an unkillable process each run. Both launchers now bound the wait by wall-clock deadline, with captured output capped at 8 MiB.
  • SMTP already bounded its submission, but a budget that expired mid-connect left the cancelled operation’s completion handler queued โ€” to be run by the retry against the next resolved address, with references into a stack frame that no longer existed. Handler state is heap-owned now, and a spent budget ends the endpoint walk instead of retrying into it. (#1471)

๐Ÿงน Injection, scrubbing and resource limits

  • Graphite status paths go through the same scrub as the perf path. The ${check_alias} substituted into them can come from a remote submitter, so an alias carrying a newline injected an extra, attacker-chosen metric line into Graphite (and a ; injected carbon tags) โ€” a way to hide a real problem or fabricate one. (#1465, #1467)
  • Inbound NSCA wire fields are validated before they reach the logs and the inbox channel: control characters are stripped from host and service names, and a return code outside 0โ€“3 is clamped to UNKNOWN instead of flowing on as an arbitrary 16-bit integer. (#1460)
  • SMTP reply text is rendered inert before it is logged. Anything outside printable US-ASCII is replaced โ€” the C0 controls and the C1 range (0x80โ€“0x9F), which carries single-byte terminal escapes such as CSI โ€” so a multi-line reply can no longer forge extra log lines. The reply to STARTTLS must be exactly 220 per RFC 3207 rather than any 2xx, and AUTH credentials containing a NUL are refused before connecting. (#1471)
  • Buffers are bounded. An NRDP response body is capped at 5 MB (previously unbounded โ€” a hostile server, or a man in the middle on a plain http:// target, could stream the agent out of memory), and an SMTP reply at 64 KB per line and 100 lines. Nothing previously capped how much a peer could make the client buffer inside its timeout window, so bytes without a line ending, or endless 250- continuations, turned a 30-second budget into gigabytes.
  • Credentials are masked in the trace log. The trace-level target dump printed raw password / token values; the fix is in the shared client machinery, so every outbound client module is covered. (#1466)
  • Unverified links are called out. An Icinga https submission whose verify mode resolves to no peer verification logs a message naming the endpoint. An empty NSCA password with encryption enabled logs an error on both ends โ€” the key is the password zero-padded with no derivation step, so an empty one is a well-known all-zero key.

๐Ÿ”Ž ElasticClient โ€” verified, authenticated, and Elasticsearch 8 compatible

Setting Default Purpose
tls version 1.2+ TLS floor for https:// addresses
verify mode peer Certificate verification (was hardcoded to none)
ca ${ca-path} CA bundle
user / password (empty) HTTP basic authentication
api key (empty) API-key authentication
timeout 30 Budget for the whole submission
event type, metrics type, nsclient log type (now empty) Legacy _type; set explicitly on ES 6.x or older

Beyond the TLS and auth work: every document in a bulk request now gets its own _id, so multi-line events show up completely instead of overwriting each other down to a single entry; responses are parsed defensively and non-2xx statuses are reported instead of ignored; a timestamp from one event line no longer leaks into later lines; and events are refused after unloadModule. (#1453)

๐Ÿ“จ SyslogClient โ€” a well-formed datagram, and options that reach the wire

Datagrams now read <PRI>TIMESTAMP HOSTNAME TAG MESSAGE. The hostname setting under [/settings/syslog/client] โ€” until now read but never used โ€” fills the HOSTNAME field (default auto, the machine name). Receivers that promoted the tag (default NSCA) to origin host will now file records under the real host name, so adjust any log-parsing rule keyed on the old, hostname-less format. An IPv6 hostname is kept intact.

tag_syntax, message_syntax and the per-state severity options take effect for the first time; an unknown severity/facility falls back to <13> instead of <0>; and all C0 control bytes and DEL in the outgoing line are replaced with spaces (previously only CR, LF and NUL), so check output cannot smuggle ANSI escape sequences into the receiver’s log. (#1470)

๐Ÿงฑ Filter framework โ€” bounded expression length and depth

A filter / warning / critical expression โ€” and a %(...) expression placeholder inside a syntax template โ€” longer than 1024 characters or nested more than 64 parentheses deep is rejected at parse time with a clear “exceeds the maximum length/depth” error. Both the recursive-descent parser and the AST evaluator recurse with the shape of the input, so an unbounded or deeply nested expression could exhaust the thread stack and crash the whole agent โ€” reachable by anyone able to influence a filter string over the authenticated REST API, or over NRPE with allow arguments = true. String literals are exempt from the depth count, and real filters are a small fraction of both limits. (#1469)

๐Ÿš CheckExternalScripts โ€” sandbox, arguments and the shell fallback

Beyond the timeout work above: the show / delete sandbox resolves symlinks before its containment test (previously a symlink inside the script root pointing outside it let an authenticated admin read or remove files anywhere the service account could reach); % and ^ are refused on the Windows shell fallback (cmd.exe %VAR% expansion and its escape character, opt-out via allow nasty characters); add arguments is honoured and list --include-lib works; and a null-provider dereference and the help-pb argument numbering are fixed. The docs now warn that write access to any script path directory is equivalent to code execution as the service account, and clarify that allow arguments does not gate aliases. (#1468)

๐Ÿ–ฅ๏ธ Web UI โ€” dependency modernization

Every dependency in web/package.json moves to its latest release: MUI (material, icons, x-charts) 7/8 โ†’ 9, react-router 7 โ†’ 8, eslint 9 โ†’ 10, TypeScript 5.9 โ†’ 6.0, plus minor bumps for react, redux, zod, vite and vitest. The UI is adapted to the MUI 9 breaking changes โ€” removed system props moved into sx, renamed outlined icons, the containedPrimary shadow expressed as a theme variant, and Autocomplete’s renderInput params now exposing slotProps.input.

Both web suites are now part of the CI build (build-web.yml, Node raised to 22 for react-router 8): 65 vitest unit tests and 18 Playwright integration tests driving the built bundle in a real Chromium. The e2e preview server binds to 127.0.0.1. (#1459)

๐Ÿงช Tests and coverage

A sweep of the gcovr reports added unit tests to every source file under 50% combined line coverage that can be exercised deterministically โ€” roughly 4,000 lines of new test code across the check_mk wire protocol, pid_file, the compat helpers, execute_process_unix, the NRDP/NSCA-ng/NSCP client handlers, Icinga target objects, the CheckDisk file filter, perf_filter, the where-engine evaluation context, the external-scripts provider, CheckSystemUnix network / service / cpu-frequency, the simple file logger, the zip plugin, onboarding’s chown_subtree and the settings proxy. New integration suites cover CheckExternalScripts commands, Elastic submission and NSCA-NG cert mode.

check_cpu_frequency was refactored to take a sysfs base path so a fixture tree can drive it; production behaviour is unchanged.

๐Ÿ› Bug fixes

  • NSClientServer: check_nt instance listing could crash the serving I/O thread. list_instance() advanced a tokenizer iterator under a guard that was always true, making the invalid-line branch dead and dereferencing tok.end() on any line with fewer than three comma-separated fields โ€” which a failed PDH enumeration produces (ERROR: โ€ฆ). It now advances to the third field explicitly and logs genuinely malformed lines. (#1463)
  • NSCA-NG scenario docs corrected. The example nsca-ng.cfg used command instead of command_file (so it did not parse) and carried an authorize block with only a password, which nsca-ng treats as authorizing nothing โ€” the PSK handshake succeeds and every submission is rejected with FAIL. The example now has anchored hosts/services patterns, a Common Gotchas entry for that symptom, a danger admonition against wildcard commands patterns, and guidance on PSK entropy and not passing --password on the command line. (#1462)
  • Elastic: dead copy-paste from the Graphite client removed (elastic_handler.hpp, the unused channel/command machinery and the client-parser dependency).
  • execute_process_w32 uses GetTickCount for XP-toolset compatibility.

โš ๏ธ Upgrade notes

  • ๐Ÿ”’ NSCA-NG cert mode really verifies now. Upgrade if any target sets use psk = false. A target whose server certificate does not chain to the configured ca, or does not match the host name, will start failing to connect โ€” fix the certificate, or accept the exposure explicitly with insecure = true. Servers requiring a client certificate will now receive one. PSK mode is unaffected.
  • ๐Ÿ”’ check_mk targets with use ssl = true now really negotiate TLS. A plaintext-only server end will start failing โ€” loudly, which is the point.
  • ๐Ÿ”’ An unrecognized NSCA encryption value is a hard error. A typo (aes-256) or an algorithm not compiled into the build used to fall back to no encryption on the end carrying it. The NSCAServer module now refuses to load and an NSCAClient submission fails, each naming the available algorithms. Breaking only for setups relying on that fallback โ€” including builds compiled without crypto++, where every cipher name degraded to plaintext. Fix the name, or set encryption = none if plaintext was intended. Default installs (aes256) are unaffected.
  • ๐Ÿ”’ An empty NSCA password with encryption enabled now logs an error on both ends. The password is the key, so an empty one is a well-known key โ€” set the same real password on both ends.
  • NSCAServer’s performance data = false is honoured again. If you relied on it while it was broken, perfdata really is dropped now.
  • ๐Ÿ”’ Elastic over https verifies certificates. Point ca at your self-signed certificate, or set verify mode = none to keep the old behaviour. On Elasticsearch 6.x or older, set event type, metrics type and nsclient log type explicitly โ€” the legacy _type parameter is no longer sent by default.
  • ๐Ÿ”’ Syslog targets: configured severities and templates now apply. Values set on a [/settings/syslog/client/targets/โ€ฆ] section were never read, so the built-in defaults always won. Review those sections for stale keys. Receivers whose parsing rules keyed on the old hostname-less datagram need adjusting โ€” records now arrive attributed to the agent’s host name instead of the tag. Syslog remains cleartext and unauthenticated: keep the path to the server on a trusted segment.
  • โฑ๏ธ Slow endpoints now fail instead of hanging. A submission to an unresponsive Icinga, NRDP, Graphite or Elastic endpoint gives up after timeout (default 30 s) โ€” raise it on that target if the endpoint is legitimately slower, or set timeout = 0 on an Icinga target if you depend on the old unbounded wait. Graphite’s timeout is now a budget for the whole submission, so a target that only completed by quietly taking longer will fail at the configured value.
  • ๐Ÿงน Graphite’s retry is no longer read. The module always made one attempt; the setting still appears in the reference (it is registered for all client modules centrally) but has no effect.
  • ๐Ÿ”’ tls version with a trailing + means “that version or later”. 1.2+ previously negotiated TLS 1.2 only; it now also permits TLS 1.3, and any is accepted as documented. This applies to the HTTP-based clients, the NRPE/NSCA clients and servers, and check_tcp. Pin an exact version (tls version = 1.2) if a peer misbehaves when TLS 1.3 is offered.
  • ๐Ÿ”’ CheckExternalScripts: re-run ext-scr install after upgrading so an argument lockdown lands on the setting the module actually reads. The default install is unaffected (arguments are off by default). Treat write access to any script path directory, and the ability to configure external-script commands, as equivalent to code execution as the service account.
  • ๐Ÿ”’ Filter expressions over 1024 characters or 64 nesting levels are rejected. Every normal configuration is far below both limits; only a pathologically large or deeply nested expression is refused.
  • ๐Ÿ”’ Credentials no longer appear in the trace log. password and token values are masked in the target dump at log level trace, across every outbound client module.

Full detail on the security items lives in Security notices; the operator actions are mirrored on Upgrading.

Download

You can download the new version from GitHub

// Michael Medin

0.18.0 Fixed some exotic passive checks, and security hardening

Most of 0.18.0 is about things that reported success while doing nothing. check_and_forward built its submission in a form no channel could read, check_nscp had counted zero crash reports since 0.4.2, and a module enabled by a fleet bundle was never loaded until the service restarted โ€” all three are fixed. Security reviews of the SMTP, NRPE, NRDP/NSCA and WEB modules landed alongside them, and the bundled OpenSSL moves to 3.5.8.

It also adds run_schedules for submitting a passive result without waiting out the interval, lets the MSI install your own TLS certificates, and gives real-time filters a way to prime their destination at startup.

โœจ Highlights

  • ๐Ÿ“ค check_and_forward submits again. The command ran the check, answered Message submitted and delivered nothing: the submission was built as a query message, which no channel can read. NSCA, NRDP, Graphite and every other client module were equally affected. It also gained channel, alias, destination and source. (#1452)
  • ๐Ÿ“… New: run_schedules. Run the configured schedules now instead of waiting out their interval, and submit the results on their normal channel โ€” nscp client --boot --query run_schedules, optionally --argument schedule=<alias>. Works over NRPE, REST and nscp test too. (#1450, #1452)
  • ๐Ÿ’ฅ check_nscp is a filter check, and its crash count works. It has read 0 crashes since 0.4.2 (it matched the extension txt against .txt), and since 0.6.10 there were no .txt reports to find. It now recognises .crash, reads the configured archive folder again, and exposes crashes, errors, uptime, crash_age, last_crash, last_error, version and date as filter keywords. (#1451)
  • ๐Ÿ›ก๏ธ Four security reviews โ€” SMTP, NRPE, NRDP/NSCA and WEB โ€” closed a set of defense-in-depth gaps: STARTTLS response injection, an unvalidated EHLO name, certificate verification that could not work on Windows, a metachar guard that ran before decoding, secrets in the trace log, and session tokens from a non-cryptographic generator.
  • ๐Ÿ” The bundled OpenSSL moves from 3.5.4 to 3.5.8 in the Windows builds, picking up four upstream security releases โ€” most relevantly CVE-2025-11187, a stack overflow parsing a hostile PKCS#12 file, reachable through check_certificate. (#1445)
  • ๐ŸชŸ The MSI can install your own TLS certificates. CERTIFICATE, CERTIFICATE_KEY and CERTIFICATE_CA place your files where every server module reads them, so the self-signed fallback is never generated. (#568)
  • โฑ๏ธ Real-time filters can prime their destination at startup. A new run on startup key submits the filter’s empty message once when the agent starts, so check_cache stops answering “Entry not found” after a restart. (#584)
  • ๐Ÿ“ก New ${address_ipv4} / ${address_ipv6*} hostname placeholders for every passive client, so a host can report itself by address instead of name. (#349)

๐Ÿ” Detailed changes

๐Ÿ“ค CheckHelpers โ€” check_and_forward delivers, and takes arguments properly

The command handed the raw QueryResponseMessage to the submission path, but channels parse a SubmitRequestMessage and the two are not wire compatible โ€” the payload sits in a different field, and field 2 of a submit message is the channel string. The channel received a message with zero payloads and cheerfully reported success. It now converts the query result into a submission first and checks the reply.

Option Meaning
channel Where to submit (default NSCA); target remains a synonym
alias Service description; defaults to the wrapped command’s name
destination Destination host for the submission
source Source host for the submission

target previously defaulted to the empty string, for which no handler exists, so even a correctly built message had nowhere to go.

๐Ÿ“… Scheduler โ€” run_schedules

run_schedules executes the schedules under [/settings/scheduler/schedules] immediately and submits each result on its own channel, target, source and alias with the same report filter, so the monitoring server cannot tell it from a timed run. The timers are untouched.

nscp client --boot --query run_schedules
nscp client --boot --query run_schedules --argument schedule=cpu

schedule= is repeatable and defaults to every schedule; an unknown alias is an error that names the ones you have. A schedule whose command is run_schedules is refused, and a reentrancy guard catches the indirect case (via check_timeout, for instance) โ€” previously that recursed until the agent died.

The caller’s identity is forwarded to the checks it runs, so REST and NRPE permissions apply to them; the scheduler’s own timed runs stay attributed to Scheduler.

๐Ÿ’ฅ CheckNSCP โ€” check_nscp rewritten as a filter check

Three independent bugs kept the crash count at zero: the extension comparison (txt vs the .txt the helper returns) has been false since 0.4.2; 0.6.10 replaced breakpad’s <guid>.dmp + .dmp.txt pair with a single <timestamp>.crash file, so even a fixed match found nothing; and 0.4.3 stopped reading [/settings/crash] archive folder, hardcoding the compile-time default. last_crash was never populated either โ€” the newest-file watermark started at the current time.

New filter keywords: crashes, errors, uptime, crash_age, last_crash, last_error, version, date. Thresholds accept duration units (crit=uptime < 5m, crit=crash_age < 7d), and a new max-unit option (default w) caps the largest unit rendered. Crash reports are a Windows concept; on Linux crashes is always 0.

โš™๏ธ Core โ€” reloads, channel verdicts and denied checks

  • A reload re-reads the included files before deciding which modules should run, so a module enabled in one since the last load is picked up. Only the includes are refreshed: clearing the whole store would discard configuration held in memory, which is exactly how nscp unit and nscp client set themselves up. One unreadable include no longer aborts the whole reload. (#1455)
  • Every channel’s verdict is reported for a channel list. All handlers were handed the same response buffer, so with channel=NSCA,GRAPHITE a failing NSCA was masked by a succeeding GRAPHITE.
  • A denied check is no longer submitted. The permission layer answers a denied query as a successful query carrying an UNKNOWN “Permission denied” payload; run_schedules and check_and_forward forwarded it, overwriting the last real result on the server while reporting success to the caller.
  • nscp client --query <cmd> no longer appends No module was specifiedโ€ฆ to every result.

๐Ÿ›ก๏ธ Security reviews

๐Ÿ”’ SMTPClient. Data pipelined across the STARTTLS handshake is refused (RFC 3207 ยง4) โ€” a prepared run of 2xx replies could otherwise walk the client through MAIL/RCPT/DATA and have it report an alert delivered while nothing was sent. The EHLO name is validated before connect, closing command injection on a relayed submission. Certificates are verified against a CA bundle through a new ca target setting and --ca argument (default ${ca-path}): the client previously used OpenSSL’s default verify paths only, which on Windows excludes the certificate store, so security=starttls failed against Gmail and M365 and operators simply turned verification off. EHLO capabilities are matched per reply line rather than by substring โ€” a greeting naming host starttls.example.com used to satisfy the STARTTLS lookup.

๐Ÿ”’ NRPE. The allow nasty characters = false guard now also runs on the decoded command and arguments; with a non-UTF-8 encoding, a multi-byte sequence could decode into a metacharacter that was never literally on the wire. A new expose version server setting (default true) lets the unauthenticated _NRPE_CHECK reply stop naming the exact build. nscp nrpe install reads the stored verify mode again instead of silently resetting it on every re-run.

๐Ÿ”’ NRDP / NSCA. A malformed <status></status> response no longer null-derefs the agent. The NRDP token and any proxy-URL credentials are redacted from the trace log โ€” including from the Target configuration: dump, which printed the raw settings map and defeated the redaction elsewhere (NSCA’s password leaked the same way). An https:// submission made through nscp client or REST with no verify mode now defaults to peer rather than trusting any certificate.

๐Ÿ”’ WEBServer. Session tokens and generated admin passwords come from OpenSSL’s CSPRNG with unbiased rejection sampling, and the server now fails closed if that RNG fails (HTTP 500 and a SECURITY: log line) rather than falling back to a weaker generator. Cookie lookups require a name boundary, so eviltoken no longer satisfies a lookup for token. Session validity and identity are read in one locked observation, closing an expiry race that could drop a request onto the anonymous grant. The web installer refuses an HTTPSโ†’HTTP redirect on the bundle download path.

๐Ÿ”’ OpenSSL 3.5.8. Windows builds only; Linux packages link the distribution’s OpenSSL. See Security notices.

๐Ÿ“ก Clients โ€” --source-host names the sender

--source-host / --sender-host were registered against the destination container, where the well-known host key is routed into the typed address field โ€” so naming a source host silently redirected the connection to it, and the sender the handler reads was never set. SMTPClient and NRDPClient had each worked around this with their own copies, which made the option ambiguous and therefore unusable on exactly the two modules most likely to need it.

๐Ÿ”ง Settings

  • settings --update --add-defaults --use-samples writes the sample objects; the flag was parsed and never read. --remove-defaults now enumerates samples too, making the two exact inverses. (#233)
  • A [/includes] entry naming a directory no longer breaks saving. The directory path reached the INI writer, and the resulting “Is a directory” error aborted the save โ€” so nscp settings --set and the web UI failed and the main file was never written. (#636)
  • New ${address_ipv4} and ${address_ipv6*} placeholders resolve in the hostname setting of NSCA, NSCANg, NRDP, Graphite, Syslog, Op5, Icinga, Elastic and Collectd clients. The address is the source address of the default route, falling back to the first non-loopback address the host name resolves to. (#349)

โฑ๏ธ Real-time filters โ€” run on startup

A new boolean filter key on the shared filter object, so it applies to CheckLogFile, CheckEventLog and the CheckSystem/CheckSystemUnix real-time filters. When true the filter submits its empty message with OK status once at startup, priming the destination. It is registered without a default on purpose: an absent key keeps the inherited value, while an explicit false overrides an inherited true. Delivery is retried across shortened waits while later modules (such as SimpleCache) are still loading. (#584)

๐ŸชŸ Windows installer โ€” install your own TLS certificates

Three new silent-install properties place your own files under the default names every server module reads, so the self-signed fallback is never generated:

Property Installed as
CERTIFICATE certificate.pem
CERTIFICATE_KEY certificate_key.pem
CERTIFICATE_CA ca.pem

The install fails if a named file is missing or is not PEM, or if a certificate is given with no key anywhere. CERTIFICATE_KEY also writes certificate key = โ€ฆ for the NRPE and WEB servers, so it is incompatible with ALLOW_CONFIGURATION=0; other servers need the setting added by hand. (#568)

๐Ÿ› Bug fixes

  • SMTP: the timeout error now says what happened in the client’s own words (timed out after 30s (the budget for the whole submission)) rather than reporting a platform error that blamed the connected party; a failed connect raises the exception callers actually catch; insecure-skip-verify is accepted over REST and no longer resets a configured target’s value on every submission; the reference no longer shows its default as N/A.
  • NRPE: the arguments-case rejection says “arguments” instead of “command”.
  • check_nscp: a crash report whose timestamp cannot be read still counts but takes no part in newest-wins, so crash_age no longer reports ~56 years.
  • Configuring with -DNSCP_BUILD_TESTS=OFF no longer aborts on the mongoose_wrapper_test target.

โš ๏ธ Upgrade notes

  • check_and_forward starts delivering, and no longer takes positional arguments. Anything that treated Message submitted as success will now see real channel failures. check_and_forward command=check_cpu warn=load>80 now fails to parse โ€” pass one arguments= per wrapped argument instead: check_and_forward command=check_cpu "arguments=warn=load>80". The positional form had to go because its parser also swallowed the CLI’s own --argument key=value tokens, which is what fed the wrapped command garbage.
  • check_nscp may start reporting CRITICAL. Its crash count works now, so an agent with an old report still in the archive folder will report a crash where it read 0. Threshold on crash_age ("crit=crash_age < 7d") if you only care about recent crashes, or clean the folder out. The message also loses its last crash: / last error: fragments โ€” put them back with an explicit detail-syntax if you match on the text.
  • A denied check now fails instead of submitting. If you restrict what a REST or NRPE identity may run, expect an error where a stale UNKNOWN previously appeared on the monitoring server.
  • NRDP over HTTPS verifies by default on the nscp client / REST path. Pass --verify none (or point --ca at the certificate) to keep submitting to a self-signed endpoint that way. Configured targets already defaulted to peer. ๐Ÿ”’
  • SMTP verifies the server certificate, against ${ca-path} by default. A target that relied on verification being effectively off needs ca pointed at the right bundle, ca = none for OpenSSL’s defaults, or insecure-skip-verify = true. ๐Ÿ”’
  • The SMTP timeout is now a budget for the whole submission, not a fresh deadline per operation โ€” a target that only completed by consuming several multiples of it will now give up.
  • --source-host no longer redirects the connection. If you used it to choose where to connect, use --host / --address.
  • settings --add-defaults --use-samples now writes samples. Drop the flag if you were relying on today’s sample-free output; --remove-defaults strips them again.
  • nscp client --query output loses its trailing No module was specifiedโ€ฆ line. Scripts that stripped it can stop.
  • Settings writes work again when [/includes] names a directory. If you removed a directory include to get saving working, you can put it back.
  • WEBServer: a script or module name beginning with - is rejected (rename it; interior dashes are fine), and POST /auth/logout now enforces allowed hosts. ๐Ÿ”’
  • NRPE: allow nasty characters = false now also inspects decoded input, so a request the guard was always meant to block may now be rejected. Set expose version = false to stop the _NRPE_CHECK ping naming your build. ๐Ÿ”’

Download

You can download the new version from GitHub

// Michael Medin

0.17.0 Windows server roles get their own checks, and check messages finally read like numbers

0.17.0 adds twelve new checks โ€” eight for IIS and Remote Desktop Services on Windows, four for the status pages of the common web servers โ€” and gives every filter check control over how it renders numbers, so 140.293GB/0.983TB can become 141.09GB/1006.85GB (or 141,09GB/1.006,85GB). Alongside that, the filter engine stops quietly doing the wrong thing: text-versus-number comparisons are numeric, fractional thresholds mean what they say, and an error inside a syntax template is reported instead of rendering a blank.

Highlights

  • Eight new Windows checks for IIS and Remote Desktop Services. The new CheckWindowsApps module covers IIS sites, application pools, worker processes and HTTP.sys request queues, plus RDS CAL licensing, session counts, per-session load and the Connection Broker counterset.
  • Four new web-server status checks. check_apache_status, check_nginx_status, check_phpfpm_status and check_tomcat_status read the vendors’ machine-readable status endpoints over HTTP(S), sharing check_http’s auth and TLS handling.
  • Check messages can be told how to render their numbers. Four new options โ€” decimals, byte-unit, decimal-separator, thousands-separator โ€” on every filter check and every real-time filter (#1428). Perfdata and thresholds are untouched.
  • Filter comparisons between text and a bare number are now numeric. filter=value > 90 no longer matches value=100 as false because “100” sorts before “90”, and 90 > value evaluates at all.
  • Fractional thresholds stop being truncated. count > 2.5 meant count > 3 and working_set > 1.5g meant 1g; both now mean what they say.
  • Host name placeholders resolve across the whole settings subsystem โ€” including attachment targets and [/includes] (#458) โ€” and are sanitized before they land in a local path ๐Ÿ”’.
  • Syslog submission works again after ten years. SyslogClient read its connection settings from the wrong place and sent nothing at all; a configured syslog target will start receiving traffic on upgrade.
  • Check-specific filter keywords that shadowed the generic summary keywords are renamed, with the old names kept as deprecated aliases.

Detailed changes

CheckWindowsApps โ€” a new module for Windows server roles

A new Windows-only module carrying IIS and Remote Desktop Services checks, built on the performance counter sets and enriched from WMI where the role’s provider is installed. The two roles share one module deliberately: every check module statically links Boost and the filter engine, so a role earns its own DLL only when it drags in a heavy or optional dependency (the way CheckMySQL carries libmariadb.dll).

Command Reports
check_iis_app_pools Per-pool state, uptime and recycles. CRITICAL by default when an auto-start pool is not running; a pool that has never started since boot surfaces as unknown rather than hiding.
check_iis_sites Per-site state, connections and uptime, plus requests_per_sec / bytes_per_sec behind averages=true. CRITICAL by default when an auto-start site is stopped.
check_iis_worker_processes Per-w3wp active and served requests, with the <pid>_<pool> instance name split into keywords. An empty set is OK โ€” idle pools spin their workers down.
check_iis_request_queues Per-queue length, rejections and age, defaulting to HTTP.sys’ 1000-request limit (warn > 800, critical > 1000).
check_rds_licenses One record per CAL key pack from Win32_TSLicenseKeyPack: total, issued and available licences. Warns at available < 10 and total > 0, critical at available = 0 and total > 0.
check_rds_sessions Active, inactive and total session counts, all three as perfdata. No default thresholds โ€” the interesting limits are per-farm.
check_rds_session_load One record per session (Console, Services, RDP-Tcp <n>) with CPU, working set and, on session hosts, RDP protocol bytes โ€” the per-user attribution check_process cannot give. sessions-only=true skips the session-0 aggregate.
check_rds_broker The Connection Broker counterset. Counter names vary between Windows Server versions, so the check enumerates whatever the counterset exposes and reports one record per counter instead of hard-coding names.

A host without the role gets a clean UNKNOWN naming the missing role, not a WMI or PDH error dump. The counter plumbing landed as a reusable gather helper that collects a set of English counter names for every instance of an object in one query, keeping the existing localized/English/index resolution fallback โ€” it was verified against live Swedish-localized counters.

CheckNet โ€” status-page checks for the common web servers

Command Endpoint Keywords
check_apache_status mod_status (?auto appended automatically) workers, requests/s, scoreboard
check_nginx_status stub_status active/reading/writing/waiting, cumulative accepts/handled/requests, derived dropped count
check_phpfpm_status FPM status page processes, listen queue, max_children_reached, slow requests; warns by default when requests queue up
check_tomcat_status manager status?XML=true (appended automatically) per-connector thread pool, request/error counters, JVM heap; defaults fire at 75%/90% pool usage

All four share check_http’s connection handling โ€” Basic auth, TLS version / verify / CA, timeout โ€” and go CRITICAL by default when the endpoint is unreachable, answers non-2xx, or serves something that is not the expected status format. Numeric parsing pins the classic locale, so a host with a decimal comma no longer truncates ReqPerSec at the decimal point.

Filter messages โ€” configurable number rendering

check_drivesize reported 140.293GB/0.983TB used: two units and six decimals in one line, with no way to change either (#1428). Every filter check now takes four options, and real-time filters take the same values as settings keys (decimals, byte unit, decimal separator, thousands separator), inheritable from the default template.

Option Effect
decimals Exactly N decimals. Default -1 keeps the historical “up to three, trailing zeros stripped”. Capped at 15.
byte-unit Pin every byte value to one unit, Bโ€ฆEB.
decimal-separator Radix character โ€” , for the European rendering.
thousands-separator Digit grouping for the integer part.
check_drivesize drive=/ show-all=true decimals=2 byte-unit=GB
OK /: 141.09GB/1006.85GB used

check_drivesize drive=/ show-all=true decimals=2 byte-unit=GB \
  decimal-separator=, thousands-separator=.
OK /: 141,09GB/1.006,85GB used

The format lives on the evaluation context, so it reaches the message only: performance data is built from the raw values and keeps its full precision and its . radix, and so does every number the filter grammar parses out of a threshold โ€” warning=used>1.5g means the same thing with a decimal comma in force.

Three defects in the byte formatter came out of this work:

  • format_bytes(used, 'gb') rendered 1.27055e-10, because the unit comparison was case sensitive against an uppercase table. Units are now case insensitive everywhere.
  • A unit that matched nothing fell out of the comparison having divided seven times, rendering value/1024^7. An unknown unit is now reported โ€” Filter processing failed: format_bytes failed: Unknown byte unit: ZB โ€” and the same check applies inside real-time filters.
  • format_bytes(value, '') failed to parse at all; the empty string literal is now accepted.

The filter/where engine โ€” comparisons that mean what they say

  • Text keyword versus bare number is numeric. A string-typed keyword compared against an unquoted number used to order lexically, or โ€” with the operands reversed โ€” fail to evaluate. Both sides now compare as numbers. This covers value/warn/crit/min/max (filter_perf, render_perf), speed (check_network), string_value (check_registry_value) and column() (check_logfile). A value that is not a number never matches; the check logs one warning naming it and stays a certain non-match, not UNKNOWN. Quoted literals keep the lexical comparison, as do like, regexp, in, keyword-specific converters (state = 'running', age > 30m) and the = 'unknown' / = 'never' sentinels.
  • Fractional numbers survive. count > 2.5 used to be rounded into the counter’s integer domain, and unit literals lost their fraction entirely (working_set > 1.5g meant 1g, uptime < 2.5h meant 2h).
  • filter_perf/render_perf/xform_perf: max and min were swapped. max read the perf-data minimum bound and min the maximum; they now read the bounds they name.
  • Template errors are reported. A function that failed inside detail-syntax or top-syntax left the placeholder empty and said nothing; the check now returns UNKNOWN with Filter processing failed: โ€ฆ.
  • perf-config’s unit: converts instead of relabelling. On byte series that do not auto-scale, unit:KB used to change the label only, shipping =1536KB for 1536 bytes. The value and the warn/crit bounds now convert. An unrecognised unit leaves the value alone rather than dividing it by 1024โท.

Filter keywords โ€” the clash with the generic summary keywords is resolved

A handful of checks registered a keyword named status, count or total โ€” the same names as the built-in summary keywords. The check-specific value won in filter/warning/critical and detail-syntax, while top-syntax and the reference documentation showed the generic one. Each now has a distinct name:

Check Old New
check_cpu, check_cpu_utilization total usage
check_battery status battery_status
check_network status, total link_status, throughput
check_os_updates count updates
check_patch_age count patches
check_pending_reboot count signals
check_printjobs status job_status
check_printqueue status printer_status
check_installed_software (Linux) status package_status
check_activation status activation_status
check_docker status container_status
check_connections count, total connections, total_connections
check_dns count records
check_http status status_message
check_shadowcopy count copies
check_disk_health total size

The old names remain as undocumented deprecated aliases with unchanged behaviour, so check_cpu "warn=total > 80" still works.

Settings โ€” host name placeholders, and where they may land

${host}, ${hostname}, ${hostname_lc}, ${hostname_uc} and ${domain} now resolve in attachment target paths and in [/includes], not only in settings urls and the url an attachment is fetched from (#458). An unknown ${...} token in a path is not an error โ€” it resolves to the installation directory โ€” so a configuration like [/attachments] ${shared-path}/${host}.ini = โ€ฆ never failed, it quietly wrote one file with the installation directory in its name.

๐Ÿ”’ Because the host name is not fully under the operator’s control (DHCP, or any local privileged process can set it), a value substituted into a path is reduced to the characters a legal RFC-952 host name can contain: anything else becomes _, and a dots-only value becomes _. Settings urls and the submit clients’ host name specs are unaffected. See Security notices.

nscp settings --migrate-to (and the REST migrate) now keeps a placeholder you pass it as-is in boot.ini while migrating into the expanded per-host file, the way --switch already did, so the template survives on a fleet-managed machine.

Clients โ€” submission paths that were quietly dead

  • Syslog. SyslogClient read its connection settings from the sender rather than the target, so address, port, facility, severity and templates were all ignored: the agent logged Undefined facility: and sent nothing. Broken since 0.4.3 (2015). CheckMKClient had the same defect on its query path.
  • SMTP. The sender’s host name was read from the wrong place, so it was always empty and the EHLO fell back to localhost. Set ehlo-hostname on the target if your mail server applies HELO/EHLO policy.
  • Short command names. A client command shorter than eight characters โ€” cpu, run โ€” answered Exception processing command line: basic_string::substr โ€ฆ instead of running, in every module built on the shared client machinery (NRPE, NSCA, NRDP, Graphite, โ€ฆ).

CheckSystem โ€” check_pending_reboot says since when

The CBS and Windows Update reboot keys exist only while their reboot is queued, so their last-write time is when the signal appeared. Two new keywords follow check_registry’s naming: written (type_date, plus written_s) and age, duration-typed so warning=pending = 1 and age > 7d reads as seven days. The default message gains (pending since <time>) when the time is known (#1415). The file-rename, computer-rename and domain-join signals carry no timestamp, so both keywords are optional: they render as unknown, compare false against every number and emit no perfdata rather than reporting a misleading value.

Data collection โ€” one bad field no longer sinks the cycle

  • WMI. Win32_Processor.LoadPercentage is occasionally NULL, and row::get_int had no case for it: the type-mismatch exception escaped half-way through the row and the collector threw away the entire cycle’s clock speeds and core counts (#1391). Optional fields can now opt into boost::none, mandatory ones fail with a clear <col> is NULL instead of localized COM text, and check_cpu_frequency renders a missing sample as no load sample rather than a fabricated 0.
  • PDH. MaxQueueItemAge on an idle, freshly started queue returns PDH_CALC_NEGATIVE_DENOMINATOR, which failed a whole gather even when the caller passed ignore_errors โ€” making check_iis_request_queues misreport the object as missing. With ignore_errors the counter is now skipped for that tick (#642, #906); the background collector and every single-counter check still throw, so they hear about an uncomputable counter instead of silently reading a default.

Windows installer and file layout

A round of fixes to the modern (ProgramData) layout introduced in 0.16.2: upgrading an enrolled host resolves every path token instead of failing; ReadLayout gets the install folder before directories resolve; CURRENT_LAYOUT is set through the public property setter; an upgrade of a modern host no longer re-creates nsclient.ini in Program Files; a %ProgramData% that cannot be resolved fails outright instead of half-applying the layout; migrated files get the destination’s ACL rather than the one they came with; resetting a renamed tree to inherited strips the explicit ACEs; and --migrate-layout legacy migrates to legacy instead of silently to modern.

Bug fixes

  • nscp settings --show --path โ€ฆ without a --key used to print nothing and exit 0; it now reports Invalid command line please use --path and --key with show and exits non-zero.
  • The settings diff behind the REST diff endpoint kept listing an edit for the lifetime of the process after it had been written, reporting a modified entry whose old value equalled its new one.
  • Only the pending markers the backend confirms are dropped on save, and staged deletions are masked in has_key.
  • Boolean option defaults render as true/false in the generated reference instead of garbage.

Documentation

Options shared by every filter check (filter, warning, top-syntax, โ€ฆ) and the generic filter keywords are now single-sourced: they fold out of each command’s reference page into one shared page, so a command’s documentation shows only what is specific to it. Runtime-stubbed Windows-only checks are marked as Windows only.

Upgrade notes

  • Syslog starts delivering. If you have a syslog target configured, check it still points where you want before upgrading โ€” it has not been delivering, and it will now. The same applies to SMTP targets, which will start announcing this host in EHLO instead of localhost.
  • Host name placeholders in paths now resolve. Check any ${host}, ${hostname} or ${domain} under [/attachments] or [/includes] and remove workarounds โ€” such a file lands somewhere new after upgrade. ๐Ÿ”’ The value is sanitized when it lands in a local path. Configurations without a host name placeholder are unaffected.
  • Number rendering is opt-in, but it is all-or-nothing per check. Leave all four options unset and messages are byte-for-byte unchanged. Set any of them and plain float keywords move onto the number format too: with decimals unset they render with up to three decimals instead of the legacy 6-significant-digit form (2.71094 โ†’ 2.711), and large values stop rendering scientific. A pipeline that matches float text in the message may need its pattern relaxed.
  • An unknown unit in format_bytes() now returns UNKNOWN instead of a quietly wrong number. A syntax string with a typo’d unit will fail until the unit is fixed.
  • perf-config unit: on plain byte series changes the metric’s magnitude. A dashboard that compensated for the old mislabelling will see the metric drop by the unit ratio; a graph flat at a near-zero value because of a misspelled unit: will jump to its real magnitude.
  • Filter comparisons against a bare number are numeric. Review any filter that deliberately relied on text ordering โ€” quote the number to keep the old behaviour.
  • Fractional thresholds change meaning. Whole-number thresholds are unchanged; expressions that already used a decimal point can behave differently.
  • max and min in filter_perf/render_perf/xform_perf were swapped. A filter that compensated needs the two names exchanged back.
  • Renamed filter keywords keep working through deprecated aliases, but three default perfdata keys change because the default perf-config names the renamed keyword: check_cpu_utilization (Linux) cpu_total โ†’ cpu_usage, check_patch_age patch_count โ†’ patch_patches, check_pending_reboot reboot_count โ†’ reboot_signals. Pass your own perf-config=extra(...) with the old name to keep the old key. check_os_updates’ default output now reports the actual number of updates instead of the matched-row count.
  • check_pending_reboot’s default message gains a suffix โ€” Reboot required: Windows Update (pending since 2026-08-16 09:41:12). Notification pipelines matching the exact message text need their pattern relaxed.
  • nscp settings --show without --key now fails. Scripts relying on the silent success need the missing --key added.

Download

You can download the new version from GitHub

// Michael Medin

0.16.4 Security release: request-smuggling fixes in the bundled web server

0.16.4 upgrades the Cesanta Mongoose web server bundled in the Windows builds to 7.23, closing two critical HTTP request-smuggling vulnerabilities in its HTTP parser. If NSClient++’s web server is reachable through a reverse proxy or WAF, upgrade promptly.

Highlights

  • ๐Ÿ”’ Bundled Mongoose upgraded from 7.20 to 7.23. Fixes two critical (CVSS 9.1) HTTP request-smuggling vulnerabilities, CVE-2026-73256 and CVE-2026-73257, fixed upstream in Mongoose 7.22.
  • Windows builds only. The Windows WEBServer module (REST API and web UI) uses the Mongoose backend; the Linux DEB/RPM packages build on Boost.Beast and never contained the vulnerable code.
  • Exploitable behind an intermediary. Both flaws let an unauthenticated attacker smuggle requests past a reverse proxy, WAF or load balancer in front of NSClient++ โ€” bypassing proxy-level ACLs or injecting into other clients’ reused connections. Direct client โ†’ NSClient++ deployments have no front end to desynchronize, and NSClient++’s own authentication is still enforced per request either way.

Detailed changes

WEBServer โ€” bundled Mongoose upgraded to 7.23 (security)

Mongoose versions before 7.22 mis-parse HTTP message framing in two ways:

CVE Flaw
CVE-2026-73256 Broken HTTP/1.0 detection in http_cb() โ€” a request combining Transfer-Encoding: chunked with conflicting HTTP/1.0 framing is parsed with different message boundaries than an HTTP/1.0 reverse proxy sees.
CVE-2026-73257 Requests carrying both Content-Length and Transfer-Encoding: chunked are accepted instead of rejected, enabling CL.TE desynchronization against a Content-Length-preferring front end.

All build pipelines now pin Mongoose 7.23 (the latest release, which also carries further upstream TLS and TCP/IP hardening): the Windows CI workflows, the Linux docker scenario images that fall back to the Mongoose backend (minimal, no-openssl), and the developer build instructions. The full advisory record is on the Security notices page.

Upgrade notes

  • ๐Ÿ”’ Upgrade Windows installs, promptly if behind a reverse proxy/WAF: the request-smuggling CVEs only matter when an intermediary in front of NSClient++ frames the HTTP stream differently than the built-in web server. No configuration change is needed โ€” this is a drop-in upgrade.
  • Linux packages are unaffected (Boost.Beast web backend, no Mongoose), as are installs with the WEBServer module disabled.

Download

You can download the new version from GitHub

// Michael Medin