Why does client authenticate before associating with the AP. What does open system authentication even achieve. I fail get an accurate reason from my web searches. Best reason I do get is unlike wired medium there is no concept of physical port in wireless medium. This makes sense but still there is one confusion: why don’t we skip authentication and just start with association? The purpose of authentication can be achieved with association itself. No?
Why does client authenticate before associating with the AP
This does seem a bit odd. Yes, in modern 802.11 there is an open authentication that takes place before the association request/response. This is also prior to any sort of 802.1X authentication or WPA/WPA2 key exchange.
However, you need to keep in mind that there was an original 802.11 standard from the IEEE. Things like 802.11i (aka WPA2) or 802.11ac are amendments to the existing standard. These amendments for the most part try to extend or enhance the base standard while leaving as much of the existing standard intact to provide backwards compatibility.
So this pre-association authentication takes place because of a legacy feature, namely WEP. This is the point where a WEP client provided it's authentication to the AP. When 802.11i was added to the standard, this was a more involved process for authentication. As such, 802.11i is performed post association, but prior to being given access to the network.
While the new process no longer made use of the pre-association authentication request/response, it was left in the process as an open authentication to provide a consistent experience for clients/APs. It simplifies the process and development if both legacy clients/APs and newer clients/APs can simply expect the authentication/association flow rather than to have to look for two different connection flows.
Note: you can find the full current connection process (with 802.1X) in this Cisco/Meraki document. The initial authentication and association process takes places for all current 802.11 connections.
However, WPA3 proposed by the WFA (which is not an IEEE standard) has for the first time changed this process with the SAE (simultaneous authentication of equals) feature. My understanding is that this changes the 802.11 authentication process from the two frame request/response to a four frame commit/confirm process.
I have yet to see a IEEE working group discussing any changes related to WPA3 at this time.
Update: I did forget to add that the 802.11 authentication is where MAC authentication takes place (mainly forgot since I don't use MAC auth). This is of particular note to modern wireless networking as vendors are adding their own updates for better control of PSKs (Cisco->Identity PSK, Aerohive->Private PSK, Ruckus->Dynamic PSK, MiKroTik->Private PSK, etc).
AFAIK, these wireless platforms all use the 802.11 authentication phase to set the PSK to use for the device in question through some mechanism (varied and non-standardized at this point), all prior to the WPA2 handshake so the WPA2-Personal experience is transparent to the client device.