All writing

~/writing/goscan-one-socket-subnet

Systems debugging
4 min read

One socket for the whole subnet

A host scanner that opened a raw socket and a goroutine per host fell over on anything bigger than a /24. The rewrite uses one socket and two goroutines no matter how many hosts, with an ARP fast path and an unprivileged fallback.

goscan does one thing: find the live hosts on every subnet on every interface, fast, and print them in a way you can pipe into the next command. The useful shape is goscan alive -i eth0 | xargs -I{} ssh root@{} uptime.

The first version worked and didn't scale. It opened a pinger, and a raw socket, per host, and spawned roughly one goroutine per host to go with it. On a /24 that's 254 sockets and 254 goroutines, which is ugly but survives. Point it at a /16 and you're asking the kernel for 65,000 raw sockets at once. It does not go well.

So I rewrote the core around a single shared socket.

probe.go
// Host discovery via a single shared ICMP raw socket per scan, with an
// optional ARP fast-path for local subnets. The old design opened one
// Pinger (and one raw socket) per host and spawned ~N goroutines; this
// one uses two goroutines (sender + receiver) and one socket regardless of N.

Two goroutines, any number of hosts

The whole sweep is one raw ICMP socket, a goroutine that sends, and a goroutine that receives. The trick that makes one socket enough is the ICMP echo sequence number: I set it to the host's index in the list, so a reply identifies its sender without me tracking per-host state.

The sender walks the host list, fires one echo per host with Seq set to the index, and paces itself with a small gap so it doesn't blast the NIC:

probe.go (the sender)
payload := []byte("goscan/v1")
gap := time.Duration(icmpSendGapNsec) // 200 microseconds
for _, idx := range pending {
    msg := icmp.Message{
        Type: ipv4.ICMPTypeEcho,
        Body: &icmp.Echo{ID: int(id), Seq: idx, Data: payload},
    }
    b, _ := msg.Marshal(nil)
    var dst net.Addr = &net.IPAddr{IP: hosts[idx].AsSlice()}
    if !privileged {
        dst = &net.UDPAddr{IP: hosts[idx].AsSlice()}
    }
    conn.WriteTo(b, dst)
    time.Sleep(gap)
}

The receiver reads replies and matches each one by sequence number straight back to the host index, with an atomic compare-and-swap so a host that answers twice is only counted once:

probe.go (the receiver)
// Primary match: sequence number == host index.
if seq := echo.Seq; seq >= 0 && seq < len(hosts) {
    if replied[seq].CompareAndSwap(false, true) {
        emit(seq, true)
    }
    continue
}
// Fallback: match by source IP (covers the rare seq mismatch
// in unprivileged mode).
if addr, ok := srcAddr(src); ok {
    if idx, ok := addrToIdx[addr]; ok && replied[idx].CompareAndSwap(false, true) {
        emit(idx, true)
    }
}

The sequence number is 16 bits, so a single sweep tops out at 65,535 hosts. That's the one hard limit (icmpMaxSubnet = 0xFFFF), and a /16 sits right under it. Bigger than that and the scan refuses rather than silently wrapping the index.

ARP gets there first

For a local subnet, ICMP is the slow way to ask "are you there." The machines on your own broadcast domain will answer ARP, and ARP needs no raw-socket privilege and no round trip through the IP stack. So for any subnet that's a /24 or smaller, goscan runs an ARP pass first, with a pool of workers, and only falls through to the ICMP sweep for the addresses ARP didn't already confirm.

probe.go
// Stage 1: ARP fast path for local subnets (/24 or smaller).
if bits >= 24 && supportsARP() {
    // j-keck/arping uses AF_PACKET on Linux and BPF on the BSDs/darwin.
    // arpWorkers = 64
}

Working without sudo

Raw ICMP normally means root. That's a bad ask for a tool you want to pipe into a shell loop, so goscan tries the privileged path and quietly drops to the unprivileged one if it can't have it. Linux exposes unprivileged datagram ICMP, gated by net.ipv4.ping_group_range, and that's enough to ping without a raw socket.

probe.go
func openICMPSocket() (*icmp.PacketConn, bool, error) {
    if c, err := icmp.ListenPacket("ip4:icmp", "0.0.0.0"); err == nil {
        return c, true, nil // privileged raw ICMP
    } else if runtime.GOOS == "windows" {
        return nil, false, err
    }
    if c, err := icmp.ListenPacket("udp4", "0.0.0.0"); err == nil {
        return c, false, nil // unprivileged datagram ICMP
    }
    return nil, false, fmt.Errorf("try sudo, or setcap cap_net_raw+ep")
}

The boolean it returns is why the receiver has that source-IP fallback. On the unprivileged datagram socket the kernel owns the ICMP id and can rewrite it, so the sequence-as-index trick is slightly less reliable there, and matching on the source address picks up the stragglers.

What it does not do

It does not enumerate IPv6. People ask, and the answer is that you cannot sweep a /64. There are eighteen quintillion addresses in one; you'd never finish, and nothing real lives at a brute-forceable offset anyway. Host discovery by sweeping is an IPv4 idea. On v6 you find hosts by listening, not by knocking on every door, so goscan stays an IPv4 tool and says as much.

The result is a scanner whose footprint doesn't grow with the subnet: one socket, two goroutines, and a sequence number doing the bookkeeping that used to take thousands of sockets.

The code is on GitHub.