An example of a line from a system log file:

Nov  5 10:54:40 intra pluto[2332]: "C2"[1] 192.168.1.200 #1: 
   responding to Main Mode from unknown peer 192.168.1.200

Errors in phase 1 usually mean an incorrect authentication configuration (e.g. incorrectly configured certificates or a different IPSec ID used) or in rare cases, incorrectly configured encryption algorithms.

At the beginning of each connection, the peers typically exchange information about their capabilities and recognize whether the connection is passing through NAT:

packet from 192.168.1.200:500: received Vendor ID payload [draft-ietf-
    ipsec-nat-t-ike-00]
packet from 192.168.1.200:500: received Vendor ID payload [draft-ietf-
    ipsec-nat-t-ike-02_n]
responding to Main Mode from unknown peer 192.168.1.200
ignoring Vendor ID payload [47bbe7c993f1fc13b4e6d0db565c68e50102010...]
ignoring Vendor ID payload [da8e937880010000]
received Vendor ID payload [Dead Peer Detection]
received Vendor ID payload [XAUTH]
NAT-Traversal: Result using draft-ietf-ipsec-nat-t-ike-02/03: no NAT detected
ignoring informational payload, type IPSEC_REPLAY_STATUS
ignoring informational payload, type IPSEC_INITIAL_CONTACT

Then the one who initiates the connection sends their certificate:

Peer ID is ID_DER_ASN1_DN: 'CN=client1'

The system checks whether the certificate is trustworthy via certification authorities. Since the Intra2net system does not use this function, it always fails:

issuer cacert not found
X.509 certificate rejected

Next, the system checks whether the certificate is known. In this example, this fails because a different certificate is configured:

no RSA public key known for 'CN=client1'

The Intra2net system therefore sends a short error message to the peer:

sending encrypted notification INVALID_KEY_INFORMATION to 192.168.1.200:500

If the Intra2net system establishes a connection on its own, we get much less information in case of the same error:

we have a cert and are sending it
ignoring informational payload, type INVALID_CERTIFICATE

In this case, the log of the peer should be examined in more detail, where more detailed information can be found.

If the peer tries to establish a connection with an incorrect authentication method, or if no connection is expected from this peer IP, the following is logged:

initial Main Mode message received on 192.168.1.254:500 but no connection 
        has been authorized with policy=PUBKEY

Instead of "policy=PUBKEY" it is possible to get "policy=PSK" or "policy=XAUTHRSASIG+XAUTHSERV" if the other party has chosen a corresponding authentication. Check the authentication methods set, the IP of the peer and the tunnel configuration, as the latter affects the IPs that are expected to connect.

If the peer wants to establish the connection with an unauthorized encryption algorithm (in this example a simple DES), the following is logged:

OAKLEY_DES_CBC is not supported.  Attribute OAKLEY_ENCRYPTION_ALGORITHM

The peer can propose multiple algorithms. If there are no acceptable ones, the Intra2net system logs this and sends a corresponding message to the peer:

no acceptable Oakley Transform
sending notification NO_PROPOSAL_CHOSEN to 192.168.1.200:500

In this case, the encryption profile on the Intra2net system or the remote peer must be adapted in such a way that at least one algorithm combination is allowed on both sides.

If everything is configured correctly, phase 1 will be completed successfully:

sent MR3, ISAKMP SA established

In phase 2, the data for the IP tunnels is negotiated. If an error occurs here, it is mostly due to incorrect IP addresses for the tunnel. However, there may not be suitable encryption algorithms here either.

Unsuitable IP addresses are logged as follows:

cannot respond to IPsec SA request because no connection is known for 
    192.168.2.0/24===192.168.1.254[CN=server-vpn]...192.168.1.200[CN=client1]

In this case, the client forgot to configure the virtual IP. This can be seen from the fact that no network is specified behind the IP address of the client. Therefore, the client wants to connect to their real IP instead of the virtual IP (which often fails due to NAT).

An attempt to connect to an incorrect virtual IP (in this case 192.168.2.78) would look like this:

cannot respond to IPsec SA request because no connection is known for 
    192.168.2.0/24===192.168.1.254[CN=server-vpn]...
    192.168.1.200[CN=client1]===192.168.2.78/32

If the peer wants to establish a connection without PFS (Perfect Forward Secrecy), but if it is activated on the Intra2net system, it looks like this in the logs:

we require PFS but Quick I1 SA specifies no GROUP_DESCRIPTION
sending encrypted notification NO_PROPOSAL_CHOSEN to 192.168.1.200:500

The encryption algorithms must also match in phase 2. If this is not the case (in the example, the client wants to encrypt with simple DES), it looks like this:

IPSec Transform [ESP_DES (64), AUTH_ALGORITHM_HMAC_SHA1] refused due 
    to insecure key_len and enc. alg. not listed in "esp" string
no acceptable Proposal in IPsec SA
sending encrypted notification NO_PROPOSAL_CHOSEN to 192.168.1.200:500

A successful connection setup, on the other hand, is logged as follows:

IPsec SA established