All writing

~/writing/goscan-one-socket-subnet

Systems debugging
4 min read

One socket for the whole subnet

A host scanner that opened a raw socket and a goroutine per host fell over on anything bigger than a /24. The rewrite uses one socket and two goroutines no matter how many hosts, with an ARP fast path and an unprivileged fallback.

goscan does one thing: find the live hosts on every subnet on every interface, fast, and print them in a way you can pipe into the next command. The useful shape is goscan alive -i eth0 | xargs -I{} ssh root@{} uptime.

The first version worked and didn’t scale. It opened a pinger and a raw socket per host, plus a goroutine per host. On a /24 that’s 254 sockets and 254 goroutines: ugly but survivable. Point it at a /16 and you’re asking the kernel for 65,000 raw sockets at once. That fails.

I rewrote the core around a single shared socket.

probe.go
// Host discovery via a single shared ICMP raw socket per scan, with an
// optional ARP fast-path for local subnets. The old design opened one
// Pinger (and one raw socket) per host and spawned ~N goroutines; this
// one uses two goroutines (sender + receiver) and one socket regardless of N.

Two goroutines, any number of hosts

The whole sweep is one raw ICMP socket, a sender goroutine, and a receiver goroutine. One socket is enough because of the ICMP echo sequence number: I set it to the host’s index in the list, so a reply identifies its sender with no per-host state.

The sender walks the host list, fires one echo per host with Seq set to the index, and paces itself with a small gap so it doesn’t blast the NIC:

probe.go (the sender)
payload := []byte("goscan/v1")
gap := time.Duration(icmpSendGapNsec) // 200 microseconds
for _, idx := range pending {
    msg := icmp.Message{
        Type: ipv4.ICMPTypeEcho,
        Body: &icmp.Echo{ID: int(id), Seq: idx, Data: payload},
    }
    b, _ := msg.Marshal(nil)
    var dst net.Addr = &net.IPAddr{IP: hosts[idx].AsSlice()}
    if !privileged {
        dst = &net.UDPAddr{IP: hosts[idx].AsSlice()}
    }
    conn.WriteTo(b, dst)
    time.Sleep(gap)
}

The receiver reads replies and matches each one by sequence number straight back to the host index, with an atomic compare-and-swap so a host that answers twice is only counted once:

probe.go (the receiver)
// Primary match: sequence number == host index.
if seq := echo.Seq; seq >= 0 && seq < len(hosts) {
    if replied[seq].CompareAndSwap(false, true) {
        emit(seq, true)
    }
    continue
}
// Fallback: match by source IP (covers the rare seq mismatch
// in unprivileged mode).
if addr, ok := srcAddr(src); ok {
    if idx, ok := addrToIdx[addr]; ok && replied[idx].CompareAndSwap(false, true) {
        emit(idx, true)
    }
}

The sequence number is 16 bits, so a single sweep tops out at 65,535 hosts. That’s the one hard limit (icmpMaxSubnet = 0xFFFF), and a /16 sits right under it. Bigger than that and the scan refuses rather than silently wrapping the index.

ARP gets there first

For a local subnet, ICMP is the slow way to ask “are you there.” Machines on your own broadcast domain answer ARP, and ARP needs no raw-socket privilege and no trip through the IP stack. So for any subnet /24 or smaller, goscan runs an ARP pass first with a pool of workers, and falls through to the ICMP sweep only for the addresses ARP didn’t confirm.

probe.go
// Stage 1: ARP fast path for local subnets (/24 or smaller).
if bits >= 24 && supportsARP() {
    // j-keck/arping uses AF_PACKET on Linux and BPF on the BSDs/darwin.
    // arpWorkers = 64
}

Working without sudo

Raw ICMP normally means root. That’s a bad ask for a tool you pipe into a shell loop, so goscan tries the privileged path and drops to the unprivileged one if it can’t have it. Linux exposes unprivileged datagram ICMP, gated by net.ipv4.ping_group_range, and that’s enough to ping without a raw socket.

probe.go
func openICMPSocket() (*icmp.PacketConn, bool, error) {
    if c, err := icmp.ListenPacket("ip4:icmp", "0.0.0.0"); err == nil {
        return c, true, nil // privileged raw ICMP
    } else if runtime.GOOS == "windows" {
        return nil, false, err
    }
    if c, err := icmp.ListenPacket("udp4", "0.0.0.0"); err == nil {
        return c, false, nil // unprivileged datagram ICMP
    }
    return nil, false, fmt.Errorf("try sudo, or setcap cap_net_raw+ep")
}

The boolean it returns is why the receiver has that source-IP fallback. On the unprivileged datagram socket the kernel owns the ICMP id and can rewrite it, so the sequence-as-index trick is slightly less reliable there, and matching on the source address picks up the stragglers.

What it does not do

It does not enumerate IPv6. People ask. You cannot sweep a /64: eighteen quintillion addresses, you’d never finish, and nothing real lives at a brute-forceable offset. Sweeping is an IPv4 idea. On v6 you find hosts by listening, not by knocking on every door, so goscan stays IPv4.

The footprint doesn’t grow with the subnet: one socket, two goroutines, and a sequence number doing the bookkeeping that used to take thousands of sockets.

The code is on GitHub.