Is an IPSec connection between 2 devices behind NAT possible?

Question

We have tens of IPSec connections between our office and customer site's. We use an extra router in the customer network (so behind NAT) to initiate the connection to our office where a PFSense router is the "network entry" (so not behind NAT). This works perfectly fine!

In one specific case we need also a direct IPSec connection between 2 customer site's. On both site's we already have an router in their network (ubiquiti so Linux). I'm allowed to open ports at one of the 2 customer locations to get it working.

After struggling a while and try to find an solution on Google i still not able to setup a working IPSec connection between those 2 locations.

---

To make it more concrete i sketch the situation we have:

      Customer A           |        Customer B
our router --- NAT --- internet --- NAT --- our router
10.130.1.0/24                              10.130.2.0/24

At customer A i forward all ports (for test purposes) to our router and i try to initiate the IPSec connection from customer B's site. I try to initiate the connection by pinging 10.130.1.1 from 10.130.2.0/24 but don't see any log entries in the charon (ipsec) log files on either site...

Is it possible to get IPSec working at all with a construction like sketched above? So with port forwarding?

==================UPDATE 1=================

I checked if NAT-traversal was enabled at both site's and it is. I use IKEv2 in case it matters.

I was also able to get some logs by setting some log-modes and get:

Oct 26 10:09:48 12[MGR] checkout IKEv2 SA with SPIs 6680fde47ccc3276_i d590c97607b14938_r
Oct 26 10:09:48 12[MGR] IKE_SA checkout not successful

IPSec configuration (from one site as they are representive to eachother):

  allow-access-to-local-interface enable
        auto-firewall-nat-exclude enable
        esp-group FOO0 {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes128
                hash sha256
            }
            proposal 2 {
                encryption aes128
                hash sha256
            }
        }
        ike-group FOO0 {
            ikev2-reauth no
            key-exchange ikev2
            lifetime 28800
            proposal 1 {
                dh-group 14
                encryption aes128
                hash sha256
            }
            proposal 2 {
                dh-group 14
                encryption aes128
                hash sha256
            }
        }
        nat-traversal enable
        site-to-site {
            peer some_entry_A.zapto.org {
                authentication {
                    id some_entry_B.zapto.org
                    mode pre-shared-secret
                    pre-shared-secret SomePSK
                    remote-id any
                }
                connection-type initiate
                default-esp-group FOO0
                description internal_IPSec
                ike-group FOO0
                ikev2-reauth inherit
                local-address any
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks enable
                    esp-group FOO0
                    local {
                        prefix 10.130.8.0/24
                    }
                    remote {
                        prefix 10.130.22.0/24
                    }
                }
            }

Answer 1

It can work.

You need to specify on both sides the local "authentication ID" and "authentication remote-id" (crossed of course).

The reason is that each device must be able to associate the incoming packets to a configuration, so it needs an ID. Since the IP addresses are modified by NAT, they cannot be used as ID and you need to provide them another ID to work with.