dhcpv4 OFFER message isn't received if empty interface

[purpleidea]

I've been working on a built-in dhcpv4 server in https://github.com/purpleidea/mgmt/
Thanks to your fine library, I don't need to do it all from scratch! (Many thanks!)
In order to learn more about the specific protocol requirements, I've been reading and trying code in: https://github.com/coredhcp/coredhcp

In any case, I start my server like this:

server, err := server4.NewServer(interface, addr, handler4())
if err != nil {
...
}
server.Serve()

As per the code, it seems that "interface" can be the empty string, eg:

dhcp/dhcpv4/server4/conn.go

Line 40 in eed709d

However, it seems that if this is the case, at least one DHCPv4 client (stock NetworkManager in modern GNU+Linux Fedora 31) doesn't seem to ever receive the OFFER message.

My logs show:

"discover":

received DHCPv4 packet: DHCPv4 Message
  opcode: BootRequest
  hwtype: Ethernet
  hopcount: 0
  transaction ID: 0x55376c1a
  num seconds: 2
  flags: Unicast (0x00)
  client IP: 0.0.0.0
  your IP: 0.0.0.0
  server IP: 0.0.0.0
  gateway IP: 0.0.0.0
  client MAC: <SECRET>
  server hostname: 
  bootfile name: 
  options:
    DHCP Message Type: DISCOVER
    Parameter Request List: Subnet Mask, Time Offset, Router, Domain Name Server, Host Name, Domain Name, Root Path, Interface MTU, Broadcast Address, Static Routing Table, Network Information Service Domain, Network Information Servers, NTP Servers, Server Identifier, DNS Domain Search List, Classless Static Route, unknown (249), unknown (252)
    Maximum DHCP Message Size: 576
    Client identifier: [<SECRET>]

"offer"

DHCPv4 Message
  opcode: BootReply
  hwtype: Ethernet
  hopcount: 0
  transaction ID: 0x55376c1a
  num seconds: 0
  flags: Unicast (0x00)
  client IP: 0.0.0.0
  your IP: 192.168.42.101
  server IP: 192.168.42.13
  gateway IP: 0.0.0.0
  client MAC: <SECRET>
  server hostname: 
  bootfile name: 
  options:
    Subnet Mask: ffffff00
    Router: 192.168.42.13
    Domain Name Server: 8.8.8.8, 1.1.1.1
    IP Addresses Lease Time: 1m0s
    DHCP Message Type: OFFER
    Server Identifier: 192.168.42.13
    Client identifier: [<SECRET>]

You'll note that I would expect the flag to be Broadcast, and I think that seems to be a bug in the DHCPv4 server used by coredhcp, but even when I set that, it still doesn't seem to get received properly.

However, if I set the specific interface (eg: to "enp0s31f6") then everything starts working!

I don't know the specific reasons why, but without knowing anything about the low-level plumbing, perhaps this broadcast bit (

dhcp/dhcpv4/server4/conn.go

Line 26 in eed709d

// Allow broadcasting.

) doesn't work when there isn't a specific interface chosen?

I'm assuming this is a bug in your NewIPv4UDPConn, but I'm happy to learn if it's something I'm doing wrong. Happy to provide anymore info or logs if necessary.

Thank you in advance!

[Natolumin]

Thanks for the report!

Can you share with us how you actually send the response packet, if possible the whole code for your handler4 function? For comparison, coredhcp's logic for DHCPv4 is here and is not obvious.
Are the logs you shared from the server or the client ?
Do you have say a pcap of what ends up happening on the wire? That would be especially useful to understand if the packets are malformed or if they just don't end up on the right interface (or aren't sent at all).

The broadcast flag is a complex matter, but the current behaviour is not a bug as far as I know. The flag only indicates a client capability, and the server only copies the flag from the client, regardless of whether the actual network packet is a broadcast or not. The meaning of the flag is explained in RFC2131§4.1; relevant excerpt:

A client that cannot receive unicast IP datagrams until its protocol
software has been configured with an IP address SHOULD set the
BROADCAST bit in the 'flags' field to 1 in any DHCPDISCOVER or
DHCPREQUEST messages that client sends. The BROADCAST bit will
provide a hint to the DHCP server and BOOTP relay agent to broadcast
any messages to the client on the client's subnet. A client that can
receive unicast IP datagrams before its protocol software has been
configured SHOULD clear the BROADCAST bit to 0. The BOOTP
clarifications document discusses the ramifications of the use of the
BROADCAST bit [21].

A server or relay agent sending or relaying a DHCP message directly
to a DHCP client (i.e., not to a relay agent specified in the
'giaddr' field) SHOULD examine the BROADCAST bit in the 'flags'
field. If this bit is set to 1, the DHCP message SHOULD be sent as
an IP broadcast using an IP broadcast address (preferably 0xffffffff)
as the IP destination address and the link-layer broadcast address as
the link-layer destination address. If the BROADCAST bit is cleared
to 0, the message SHOULD be sent as an IP unicast to the IP address
specified in the 'yiaddr' field and the link-layer address specified
in the 'chaddr' field. If unicasting is not possible, the message
MAY be sent as an IP broadcast using an IP broadcast address
(preferably 0xffffffff) as the IP destination address and the link-
layer broadcast address as the link-layer destination address.

[purpleidea]

On Sat, Mar 28, 2020 at 6:04 AM Anatole Denis ***@***.***> wrote: Thanks for the report! Can you share with us how you actually send the response packet, if possible the whole code for your handler4 function? For comparison, coredhcp's logic for DHCPv4 is here and is not obvious.

I'm literally using the exact same code, except wrapped inside of my own mgmt wrappers. I will finish the patch shortly and publish it in its entirety.

Are the logs you shared from the server or the client ?

Sorry-- those are from the server.

Do you have say a pcap of what ends up happening on the wire? That would be especially useful to understand if the packets are malformed or if they just don't end up on the right interface (or aren't sent at all).

Fair! I haven't dug this deep yet, and for all I know it could be a bug in NetworkManager, but it's sufficiently common that I figured it would be easy to reproduce on your end, sorry.

The broadcast flag is a complex matter, but the current behaviour is not a bug as far as I know. The flag only indicates a client capability, and the server only copies the flag from the client, regardless of whether the actual network packet is a broadcast or not. The meaning of the flag is explained in RFC2131§4.1; relevant excerpt:

Yeah, I think I mis-understood part of this initially, so ignore my comments about that. The rest of the bug stands. Thanks for the excerpt though! TL;DR: If you run a basic "coredhcp" DHCPv4 server, does it work for you when interface field is empty string? Since all the examples are "TODO: dhcpv4" I assumed this was just overlooked and it's a legitimate bug. Thanks for the prompt reply!

[purpleidea]

TL;DR: If you run a basic "coredhcp" DHCPv4 server, does it work for you when interface field is empty string?

Any chance you had a chance to try? If not, lmk and I can make you a custom build.

[Natolumin]

Hey! Sorry I skipped over that question in the first read. I have a hunch:

Do you have a default route where the server is running ?

If I don't have a default route in the netns where coredhcp is running, I have the same symptoms as you: running when bound to a specified interface works fine, while running without binding to an interface (with interface = "") results in the following error:

INFO coredhcp: MainHandler4: conn.Write to 255.255.255.255:68 failed: write udp 0.0.0.0:67->255.255.255.255:68: sendto: network is unreachable

However, once I add a default route to the netns where coredhcp runs, it works fine both with and without an interface specified