Every POP gets the same monitoring build, because the worst time to design monitoring is during an outage. This is the stack, the sizing, and the alerts that actually fire on something real.
The stack#
| Component | Role | Container sizing |
|---|---|---|
| LibreNMS | SNMP polling, device inventory, alerting | 4 vCPU, 8 GB RAM, 100 GB |
| Grafana | Dashboards for NOC screens and management | 2 vCPU, 2 GB RAM, 20 GB |
| SmokePing | Latency and packet-loss baselines | 1 vCPU, 1 GB RAM, 20 GB |
| Uptime Kuma | Public-facing service checks and status page | 1 vCPU, 1 GB RAM, 10 GB |
All four run as unprivileged LXC containers on a single Proxmox VE node. At around 800 monitored devices the node sits near 6% CPU — the constraint is disk I/O from RRD writes, not compute. Put the LibreNMS container on SSD.
Build order that avoids rework#
- Proxmox host: separate storage for containers and backups, ZFS if you have the RAM for it, and a management VLAN that is reachable when the customer-facing network is not.
- LibreNMS first, with SNMP v3 (auth + priv) — v2c community strings on a public-facing edge are a finding waiting to happen.
- SmokePing pointing at your upstream gateways, a public anchor like 1.1.1.1, and every tower router.
- Grafana last, reading from LibreNMS. Do not build dashboards before you know what breaks.
SNMP on RouterOS#
/snmp communityadd name=nms-v3 authentication-protocol=SHA1 encryption-protocol=AES \authentication-password=<auth> encryption-password=<priv> \security=private addresses=10.60.0.0/24/snmpset enabled=yes contact="noc@example.net" location="POP-A"
Restricting addresses to the monitoring subnet is the difference between monitoring and an information-disclosure hole.
Alerts worth waking up for#
Most default alert sets are noise. These are the rules that earn their place:
- Device down for 3 polls — not 1. A single missed poll is usually the poller, not the device.
- Interface utilisation > 80% for 15 minutes — capacity planning signal, not an emergency page.
- Uplink packet loss > 2% for 5 minutes — this is the one that catches upstream problems before the phones ring.
- PPPoE session count drops > 20% in 10 minutes — a tower or BNG problem, visible before individual complaints.
- Optical RX power drift > 3 dB from baseline — a splice or connector degrading; fix it in daylight rather than at 2 am.
- Proxmox host disk > 85% — because the monitoring dying silently is the worst failure of all.
Route them by severity: chat for informational, phone call only for the ones that mean customers are offline.
Retention and backups#
RRD data compacts on its own; the database does not. Keep 90 days of syslog and eventlog, and back up the LibreNMS database plus /opt/librenms/config.php nightly to a different physical box. Snapshot the containers weekly from Proxmox.
Test a restore once. A backup you have never restored is a belief, not a backup.
Start smaller than this#
If you are deploying your first POP, run LibreNMS and SmokePing only. Add Grafana when someone asks for a wall display, and Uptime Kuma when you want a public status page. A monitoring stack you understand end to end beats a bigger one you inherited and never verified.
