I am reading about IPsec and am confused of what exactly is "IPsec", is it a network layer protocol, or is it a technology that uses involves multiple protocols? All web searches tell me IPsec is a "protocol" that runs on the network layer, if so, how can IPsec runs "over" UDP port 500, since UDP is on the transport layer that is above the network layer?

When look at IPsec traces such as https://www.cloudshark.org/captures/767a93d720ad, the ISAKMP is indeed above UDP, making it an application layer protocol.

If all the negotiation/cipher exchange are done on the application layer, I can't really see why we say IPsec is an network layer protocol, and what exactly does "IPsec over UPD" or "IPsec over TCP" means?

  • This is not a security question but a discussion how technologies like IPSec fit in abstract models like OSI or TCP/IP. These are only models which have their limitations. You might ask similar questions for any IP over anything technology, for things like VPN tunneled over Websockets etc. But as a perspective - you might not look at what IPSec is transported over but what IPSec provides, i.e. it provides a network layer to any other layers above. Commented Jul 5, 2023 at 4:24
  • I maybe being nitpicking, but I am a teacher and I need to know all the details to tell my students the correct thing. I am asking this question because I want to know if there is any hidden facts that I don't know, which makes IPsec a real network layer protocol. From the answers I see, the major functional part still works on Application. Thanks!
    – SamTest
    Commented Jul 5, 2023 at 11:39
  • IPSec is just "IP over something" with encryption and the encryption part does not matter for your question. So again, this is not a security question and thus off-topic here. Commented Jul 5, 2023 at 14:51
  • 2
    @SamTest: I'm not sure you actually want to know the answer, because you keep repeating your initial thoughts when the replies here told you the exact opposite. Remember what I said? I told you that IPSec is a collection of protocols, and the main protocols (ESP and AH) are network-layer protocols. Only the auxiliary protocol ISAKMP might be considered application-level. Of course you're free to disagree, but asking for an answer and then completely ignoring it doesn't seem very productive.
    – Ja1024
    Commented Jul 6, 2023 at 3:00
  • @Ja1024 I am not disagreeing. As I said, I am a teacher, so when I tell students that "IPsec is a network layer protocol", I need to explain why a network layer protocol "works on UPD port 500". I am just saying it is not strictly correct to say "IPsec is a network layer protocol" but I get what it means, and I am gonna explain it to my students.
    – SamTest
    Commented Jul 6, 2023 at 18:15

1 Answer 1


IPSec is a collection of multiple protocols. The purpose of IPSec -- the reason why it even exists -- is to add confidentiality, integrity and authenticity to the IP protocol. It does this through two protocols: the IP Encapsulating Security Payload (ESP) protocol and the IP Authentication Header (AH) protocol. When people talk about IPSec, they usually mean ESP and AH. Both are network-layer protocols which deal with IP packets.

Since ESP and AH require both parties to exchange keys, IPSec also includes the Internet Security Association and Key Management Protocol (ISAKMP) for this purpose. ISAKMP indeed uses TCP or UDP. However, this is only part of the IPSec suite. I'm also not sure why you think that ISAKMP is automatically an application-layer protocol when it uses UDP. It could also be argued that it's a session-layer protocol (layer 5).

  • Hi, Ja1024, in the real-world TCP/IP 5-layer stack, there is no session layer. Above the transport layer is application. So, if a protocol uses UDP to encapsulate its data, it should be considered as an application layer protocol, right?
    – SamTest
    Commented Jul 4, 2023 at 23:44
  • Sure, in the simplified model, this makes sense for ISAKMP.
    – Ja1024
    Commented Jul 5, 2023 at 3:41
  • 1
    @SamTest: A tunnel doesn't fit neatly in one layer. To the outside network, it's an application, but provides a data link for a network that exists inside.
    – Ben Voigt
    Commented Jul 5, 2023 at 21:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .