I have two routers and the tunnel between them is up and working well. Router1 has GRE/IP as the protocol on its tunnel interface, and as for Router2 the tunnel interface is working with IPSEC/IP. Can someone throw a theoretical insight as to how this is possible. My limited knowledge of the subject tells me that both ends need to have the same protocols configured.
No. A tunnel requires both endpoints to be able to decapsulate the traffic from the other endpoint, and one endpoint configured with one type of encapsulation will not know how to do that for a different encapsulation.
You could have two separate tunnels with different encapsulations, or you could have one tunnel inside another tunnel, e.g. GRE inside IPsec.