svilme

Base#

TCP/IP Network Model#

1. Application Layer: Responsible for providing a set of applications to users.

   HTTP, FTP, Telnet, DNS, SMTP, etc. The application layer operates in user mode.

2. Transport Layer: Responsible for end-to-end communication.

   TCP: flow control, timeout retransmission, congestion control, etc. / UDP

   MSS (Maximum Segment Size) -> TCP Segment

3. Network Layer: Responsible for encapsulation, fragmentation, routing, and forwarding of network packets.

   IP Protocol: addressing, routing. / ICMP

   MTU (Maximum Transmission Unit) -> Fragmentation

4. Link Layer: Responsible for the transmission of network packets in the physical network, such as framing network packets, MAC addressing, error detection, and transmitting network frames through network cards, etc. / ARP

OSI Seven-Layer Network Model#

• Application Layer, responsible for providing a unified interface for applications;
• Presentation Layer, responsible for converting data into a format compatible with another system;
• Session Layer, responsible for establishing, managing, and terminating communication sessions between presentation layer entities;
• Transport Layer, responsible for end-to-end data transmission;
• Network Layer, responsible for routing, forwarding, and fragmentation of data;
• Data Link Layer, responsible for framing data and error detection, as well as MAC addressing;
• Physical Layer, responsible for transmitting data frames in the physical network;

From URL to Web Page Display#

1. Parse URL -> web server and file name
2. Generate HTTP request
3. DNS domain name resolution
4. TCP three-way handshake to establish connection
5. TCP message generation
6. IP message generation -> IP address responsible for remote location
7. MAC message generation (using ARP to query MAC address) -> MAC address responsible for point-to-point transmission
8. Network card adds headers and start frame delimiters at the beginning, adds frame check sequence (FCS) for error detection at the end, digital information -> electrical signals
9. Switch (Layer 2 network device) checks the switch's MAC address table (MAC-port), if not found, forwards to all ports
10. Router (Layer 3 network device) checks the routing table for IP (route matching), ARP checks MAC.
11. Arrive at the server, parse the data packet, send HTTP response
12. Client receives HTTP response, renders the page
13. TCP four-way handshake to disconnect

During the forwarding process, the source IP address and destination IP address remain unchanged, while the source MAC and destination MAC change.

Overall, it can be divided into the following steps:

1. Enter the specified web page's URL in the browser.
2. The browser obtains the IP address corresponding to the domain name through the DNS protocol.
3. The browser initiates a TCP connection request to the target server based on the IP address and port number.
4. The browser sends an HTTP request message to the server over the TCP connection, requesting the content of the web page.
5. After receiving the HTTP request message, the server processes the request and returns an HTTP response message to the browser.
6. After the browser receives the HTTP response message, it parses the HTML code in the response body, renders the structure and style of the web page, and based on the URLs of other resources in the HTML (such as images, CSS, JS, etc.), initiates HTTP requests again to obtain the content of these resources until the web page is fully loaded and displayed.
7. The browser can actively close the TCP connection when it does not need to communicate with the server, or wait for the server's close request.

HTTP_Interview#

HTTP is the Hypertext Transfer Protocol.

What is a Protocol#

A protocol defines the format and order of messages exchanged between two or more communication entities, as well as the actions taken when sending and/or receiving a message or other events.

It establishes a specification for communication between computers using a language that computers can understand (involving two or more participants), along with various controls and error handling methods (behavior agreements and specifications).

HTTP Status Codes#

• 1xx: Indicates an informational response
  • 101 indicates switching protocols, commonly seen in WebSocket connections;
• 2xx: Indicates a successful response
  • "200 OK" indicates success
  • "204 No Content" indicates the response has no body data
  • "206 Partial Content" indicates only part of the content was sent;
• 3xx: Indicates a redirection response
  • "301 Moved Permanently" indicates permanent redirection
  • "302 Found" indicates temporary redirection
  • 303 indicates the client should resend the request along the specified path
  • "304 Not Modified" indicates the resource has not been modified, redirecting to the cache (cache redirection);
• 4xx: Indicates an error response due to client issues
  • "400 Bad Request" indicates there is an error in the client's request message
  • "403 Forbidden" indicates the server forbids access to the resource
  • "404 Not Found" indicates the specified path does not exist;
• 5xx: Indicates an error response due to server issues:
  • "500 Internal Server Error" indicates an internal server fault
  • "501 Not Implemented" indicates the functionality requested by the client is not supported yet
  • "502 Bad Gateway" indicates the server is functioning normally, but there was an error accessing the backend server
  • "503 Service Unavailable" indicates the server is busy and temporarily unable to respond.

Common HTTP Fields#

The format of an HTTP request is fixed, consisting of HTTP Header and HTTP Body. The first line is always request method path HTTP version, for example, GET / HTTP/1.1 indicates a GET request, the path is /, and the version is HTTP/1.1.

Each subsequent line follows the fixed Header: Value format, referred to as HTTP Header. The server relies on certain specific headers to identify client requests, such as:

• Host: Indicates the requested domain name; since a single server may host multiple websites, it is necessary to rely on Host to identify which website the request is for;
• User-Agent: Indicates the client's identification information; different browsers have different identifiers, and the server uses User-Agent to determine whether the client is IE or Chrome, Firefox, or a Python crawler;
• Accept: Indicates the HTTP response formats the client can handle; */* indicates any format, text/* indicates any text, image/png indicates PNG format images;
• Accept-Language: Indicates the languages accepted by the client, with multiple languages sorted by priority; the server uses this field to return a specific language version of the web page to the user.

If it is a GET request, then the HTTP request only has HTTP Header, without HTTP Body. If it is a POST request, then the HTTP request includes a Body, separated by an empty line.

POST requests typically set Content-Type to indicate the type of the Body and Content-Length to indicate the length of the Body, allowing the server to respond correctly based on the request's Header and Body.

HTTP responses also consist of Header and Body. The first line of the response is always HTTP version response code response description, for example, HTTP/1.1 200 OK indicates the version is HTTP/1.1, the response code is 200, and the response description is OK. The client relies solely on the response code to determine whether the HTTP response is successful. HTTP has fixed response codes.

The HTTP protocol uses carriage return and line feed as boundaries for HTTP headers, and the Content-Length field as the boundary for the HTTP body; both methods are to solve the "sticky packet" problem.

Other fields:

• Connection: Most commonly used for the client to request the server to use the "HTTP persistent connection" mechanism for reusing other requests.
• Content-Encoding: Indicates the compression method of the data. It specifies the compression format used for the data returned by the server.

GET vs POST#

The semantics of GET is to retrieve the specified resource from the server. The parameters of a GET request are generally written in the URL, which only supports ASCII, so GET request parameters are only allowed to be ASCII characters, and browsers impose a limit on the length of URLs (the HTTP protocol itself does not specify any limit on URL length).

The semantics of POST is to process the specified resource based on the request payload (message body). The data carried by a POST request is generally written in the message body, and the data in the body can be of any format as long as the client and server agree, and browsers do not impose limits on the size of the body.

• In the HTTP protocol, "safe" means that the request method does not "destroy" resources on the server.
• "Idempotent" means that executing the same operation multiple times results in the "same" outcome.

Thus, GET methods are safe and idempotent because they are "read-only" operations and can be cached; POST methods are neither safe nor idempotent because they modify resources on the server and are mostly non-cacheable.

HTTP Caching#

Strong Caching: As long as the browser determines that the cache is not expired, it directly uses the browser's local cache; the initiative to decide whether to use the cache lies with the browser.

• Cache-Control, is a relative time;
• Expires, is an absolute time;

Negotiated Caching: After negotiating with the server, the result of the negotiation is used to determine whether to use local cache. HTTP 200 OK / HTTP 304

• Implemented based on unique identifiers: Etag (unique identifier for the response resource) in the response header and If-None-Match in the request header;
• Implemented based on time: Last-Modified in the response header and If-Modified-Since in the request header;

Note that both fields for negotiated caching need to be used in conjunction with the Cache-Control field in strong caching; only when strong caching fails to hit can a request with negotiated caching fields be initiated.

HTTP Features (HTTP/1.1)#

Advantages and Disadvantages of HTTP#

The advantages of HTTP are "simple, flexible, easy to extend, widely used, and cross-platform."

The disadvantages of HTTP are "stateless, plaintext transmission," (plaintext transmission + inability to verify message integrity) -> "insecure."

HTTP Performance#

The HTTP protocol is based on TCP/IP and uses a "request-response" communication model:

1. Persistent connections: As long as neither end explicitly requests to disconnect, the TCP connection state is maintained, reducing the overhead caused by repeatedly establishing and disconnecting TCP connections, alleviating the load on the server side.
2. Pipelined network transmission: In the same TCP connection, after the client sends a request, it can send the next request without waiting for a response, reducing overall response time. However, the server must send responses to these pipelined requests in the order they were received.
3. Head-of-line blocking: If the server takes a long time to process request A, subsequent requests will be blocked, which is called "head-of-line blocking." HTTP/1.1 pipelining solves the head-of-line blocking for requests but does not solve the head-of-line blocking for responses.

The performance of HTTP/1.1 is average, while subsequent HTTP/2 and HTTP/3 are optimized for HTTP performance.

HTTP vs HTTPS#

1. When establishing a connection, HTTPS has an additional TLS handshake process compared to HTTP;
2. During content transmission, HTTPS encrypts the data, usually using symmetric encryption;

Differences Between HTTP and HTTPS#

• Security
• Connection establishment
• Default port
• URL prefix
• Certificate

Security Issues Addressed by HTTPS#

HTTPS adds the SSL/TLS protocol between HTTP and TCP layers.

• Eavesdropping risk -> Information encryption -> Hybrid encryption

• Tampering risk -> Verification mechanism -> Hash algorithm + Digital signature

  • Hash algorithms can ensure message integrity;
  • Digital signatures can ensure the reliability of the message source (confirming that the message was sent by the party holding the private key);

• Impersonation risk -> Identity certificate -> Embedding the server's public key in the digital certificate

  CA signs the digital certificate to ensure that the certificate is trustworthy. (Certificate trust chain issue)

  Essentially: the issue of client trust in the server

Establishing HTTPS Connection#

Basic process of SSL/TLS protocol:

• The client requests and verifies the server's public key.
• Both parties negotiate to generate a "session key."
• Both parties use the "session key" for encrypted communication.

The first two steps constitute the SSL/TLS establishment process, which is the TLS handshake phase, involving four communications (SSL/TLS 1.2).

Using different key exchange algorithms, the TLS handshake process varies; currently, two commonly used key exchange algorithms are RSA and ECDHE.

1. ClientHello, initiated by the client to request encrypted communication

   Client random number (Client Random) ...

2. ServerHello, after receiving the client's request, the server responds to the client

   Server random number (Server Random) + server's digital certificate ...

3. Client responds

   Public key encrypts random number pre-master key + client handshake completion notification ...

4. Server's final response

   Server handshake completion notification + handshake summary

At this point, the entire TLS handshake phase ends, and the client and server enter encrypted communication, which is entirely using the ordinary HTTP protocol, except that the content is encrypted with the "session key."

Ensuring Data Integrity in HTTPS#

TLS is implemented in two layers: handshake protocol and record protocol:

• The TLS handshake protocol is the process of the TLS four-way handshake, responsible for negotiating encryption algorithms and generating symmetric keys;
• The TLS record protocol is responsible for protecting application data and verifying its integrity and origin, using the record protocol to encrypt HTTP data;

The TLS record protocol mainly handles the compression, encryption, and authentication of messages (HTTP data):

• Splitting messages into multiple shorter segments and compressing each segment separately
• Adding a message authentication code (MAC, obtained through a hash algorithm) to the compressed segments to ensure data integrity and perform data authentication.
• Encrypting the compressed message segments + message authentication code using the symmetric key
• The encrypted data, along with the data type, version number, and length of the compressed data, forms the final message data.

After the record protocol completes, the final message data is passed to the transport control protocol (TCP) layer for transmission.

Evolution of HTTP#

HTTP/1.1 vs. HTTP/1.0#

• Persistent connections
• Supports pipelined network transmission, allowing subsequent requests to be sent without waiting for responses

HTTP/2.0 vs. HTTP/1.1#

• HTTP/2.0 is based on HTTPS, providing higher security
• Header compression using the HPACK algorithm
• Binary format
• Concurrent transmission, Stream, multiple requests reuse a single TCP connection
• Server push

HTTP/2.0 still faces head-of-line blocking issues, which occur at the TCP layer; only when TCP data is continuous can the application layer read data from the kernel. Once packet loss occurs, all HTTP requests will be blocked.

HTTP/3#

HTTP/3 uses the UDP protocol at the lower layer to solve head-of-line blocking issues.

The QUIC protocol based on UDP can achieve reliable transmission similar to TCP. The QUIC protocol has the following characteristics:

• No head-of-line blocking: When a packet is lost in a stream, only that stream is blocked, and it does not affect other streams; multiple Streams are independent of each other.
• Faster connection establishment: QUIC includes TLS internally, allowing connection establishment and key negotiation to be completed in just 1 RTT.
• Connection migration: The QUIC protocol does not use a four-tuple to "bind" connections but marks the two endpoints of communication using a connection ID.
• Security: In HTTP/2.0, TLS is used to encrypt and authenticate the entire HTTP session, including all HTTP headers and data payloads. TLS operates above the TCP layer, encrypting application layer data transmitted over TCP connections, but does not encrypt the TCP header and TLS record layer header, so during transmission, the TCP header may be tampered with by attackers to interfere with communication. In contrast, HTTP/3.0's QUIC encrypts and authenticates the entire data packet (including headers and body), ensuring security.

Thus, QUIC is a pseudo TCP + TLS + HTTP/2 multiplexing protocol built on UDP.

TCP_Interview#

TCP Header Format#

The TCP header occupies 20 bytes when no options are used:

• Source port number (16 bits) + Destination port number (16 bits)
• Sequence number (32 bits): solves the out-of-order problem
• Acknowledgment number (32 bits): solves the packet loss problem
• Header length (4 bits) + Reserved (6 bits) + Control bits (6 bits) + Window size (16 bits)
• Checksum (16 bits) + Urgent pointer (16 bits)

The Significance of TCP#

The IP protocol at the network layer does not guarantee reliable delivery; if reliable delivery of network packets is required, it must be handled by the TCP protocol at the upper layer (transport layer).

The TCP protocol is a reliable data transmission service operating at the transport layer, ensuring that received network packets are undamaged, uninterrupted, non-redundant, and in order.

What is TCP#

TCP is a connection-oriented, reliable, byte-stream-based transport layer communication protocol.

What is a TCP Connection#

A connection is a combination of certain state information used to ensure reliability and flow control, including socket, sequence number, and window size.

Establishing a TCP connection requires the client and server to reach a consensus on the following information:

• Socket: TCP four-tuple
• Sequence number: used to solve the out-of-order problem
• Window size: used for flow control

Differences Between TCP and UDP#

The UDP header is fixed at 8 bytes:

• Source port number (16 bits) + Destination port number (16 bits)
• Packet length (16 bits) + Checksum (16 bits)

UDP only uses the connectionless communication service provided by IP.

The differences between TCP and UDP can be summarized as follows (remember the logic):

1. Connection: TCP is connection-oriented, while UDP is connectionless.
2. Service object: TCP is a point-to-point service, while UDP can be one-to-many.
3. Reliability: TCP reliably delivers data, while UDP delivers data with maximum effort but does not guarantee reliable delivery.
4. TCP provides congestion control and flow control mechanisms to ensure data transmission security, while UDP does not.
5. Header overhead: TCP header is 20 bytes when no options are used, while UDP header is fixed at 8 bytes.
6. Transmission method: TCP is a byte-stream-based transmission, while UDP sends data in packets, maintaining boundaries.
7. Fragmentation: TCP data larger than MSS will be fragmented at the transport layer, while UDP only uses IP protocol fragmentation (greater than MTU).

TCP application scenarios:

• FTP file transfer
• HTTP/HTTPS

UDP application scenarios:

• Multimedia communication
• Broadcast communication
• Communication with smaller packet sizes: such as DNS, SNMP, etc.

Establishing a TCP Connection#

TCP establishes a connection through a three-way handshake.

Why Three-Way Handshake Instead of Two or Four?#

1. Prevents the initialization of duplicate historical connections

   "Old SYN packets" are called historical connections; the primary reason TCP uses a three-way handshake to establish connections is to prevent the initialization of "historical connections."

2. Synchronizes the initial sequence numbers of both parties

   When the client sends a SYN packet containing the "initial sequence number," the server must respond with an ACK acknowledgment packet, indicating that the client's SYN packet has been successfully received. When the server sends the "initial sequence number" to the client, it must also receive an acknowledgment from the client. This back-and-forth ensures that both parties' initial sequence numbers can be reliably synchronized.

3. Avoids resource waste

   If there were only "two-way handshakes," and the client's SYN packet got blocked in the network, causing multiple SYN packets to be sent, the server would establish multiple redundant invalid connections upon receiving the request, leading to unnecessary resource waste.

Why Are the Initialized Sequence Numbers Different When Establishing a TCP Connection?#

1. To prevent historical packets from being incorrectly received by connections with the same four-tuple (main aspect).
2. To prevent hackers from forging TCP packets with the same sequence number.

The initialization sequence number ISN random generation algorithm: ISN = M + F(localhost, localport, remotehost, remoteport).

• M is a timer that increments by 1 every 4 microseconds.
• F is a hash algorithm that generates a random value based on the source IP, destination IP, source port, and destination port. The hash algorithm must not be easily deducible from the outside; using the MD5 algorithm is a good choice.

The random number is incremented based on the clock timer, making it nearly impossible to generate the same initialization sequence number.

About SYN Attacks#

Assuming an attacker forges SYN packets from different IP addresses in a short time, the server enters the SYN_RCVD state for each received SYN packet. However, the ACK + SYN packets sent by the server cannot receive ACK responses from unknown IP hosts. Over time, this will fill the server's half-connection queue, preventing the server from serving normal users.

The most direct manifestation of SYN attack is to fill the TCP half-connection queue, causing subsequent SYN packets to be discarded when the queue is full, preventing the client from establishing a connection with the server.

To avoid SYN attacks, the following four methods can be employed:

• Increase netdev_max_backlog;
• Increase the TCP half-connection queue;
• Enable tcp_syncookies;
• Reduce the number of SYN+ACK retransmissions.

Disconnecting a TCP Connection#

TCP disconnects a connection through a four-way handshake, requiring one FIN and one ACK for each direction.

About TIME_WAIT State#

The TIME_WAIT state is necessary for two main reasons:

• To prevent data from historical connections from being incorrectly received by subsequent connections with the same four-tuple;

  The TIME_WAIT state lasts for 2MSL, which is sufficient to allow packets from both directions to be discarded, ensuring that any packets from the original connection naturally disappear from the network, and any subsequent packets are definitely from a newly established connection. (MSL is Maximum Segment Lifetime, the maximum lifetime of a packet.)

• To ensure that the party "passively closing the connection" can be correctly closed;

  The TIME_WAIT state serves to wait long enough to ensure that the final ACK can be received by the party that passively closed the connection, thus helping it to close properly.

  Suppose the client does not enter the TIME_WAIT state and directly goes to the CLOSE state after sending the last ACK packet. If that ACK packet is lost, the server will retransmit the FIN packet, and at this point, the client has already entered the closed state. Upon receiving the server's retransmitted FIN packet, it will respond with a RST packet.

  To prevent this situation, the client must wait long enough to ensure that the server receives the ACK. If the server does not receive the ACK, it will trigger the TCP retransmission mechanism, and the server will resend a FIN packet, resulting in a back-and-forth exchange that takes exactly two MSLs.

  When the client receives the server's retransmitted FIN packet, the waiting time in the TIME_WAIT state will reset to 2MSL.

Excessive TIME_WAIT states can pose two main hazards:

• First, they consume system resources, such as file descriptors, memory resources, CPU resources, thread resources, etc.;
• Second, they consume port resources, which are also limited; the generally available port range is 32768–61000, which can also be specified through the net.ipv4.ip_local_port_range parameter.

TIME_WAIT is our friend; it is beneficial to us. Do not try to avoid this state; instead, understand it.

If the server wants to avoid excessive TIME_WAIT states, it should never actively disconnect connections, allowing clients to handle TIME_WAIT instead.

If the server experiences a large number of TIME_WAIT states, it indicates that the server has actively disconnected many TCP connections, which can occur in the following scenarios:

• The first scenario: HTTP does not use persistent connections (Keep-Alive)
• The second scenario: HTTP persistent connection timeout
• The third scenario: The number of requests for HTTP persistent connections reaches the limit

TCP Keep-Alive Mechanism#

A time period is defined, and if there is no connection-related activity during this time, the TCP keep-alive mechanism will start to take effect, sending a probe packet at regular intervals. This probe packet contains very little data, and if several consecutive probe packets do not receive a response, the current TCP connection is considered dead, and the system kernel will notify the upper application.

The detection time for the TCP keep-alive mechanism is somewhat long; we can implement a heartbeat mechanism at the application layer.

Socket Programming#

For TCP socket programming:

• The server and client initialize the socket to obtain a file descriptor;
• The server calls bind to bind the socket to a specified IP address and port;
• The server calls listen to start listening;
• The server calls accept to wait for client connections;
• The client calls connect to initiate a connection request to the server's address and port;
• The server's accept returns the file descriptor for the socket used for transmission;
• The client calls write to send data; the server calls read to read data;
• When the client disconnects, it calls close, and the server will read EOF when reading data, and after processing the data, the server calls close to indicate that the connection is closed.

When the server calls accept, if the connection is successful, it will return a socket for the completed connection, which is used for data transmission. The listening socket and the socket used for actual data transmission are "two" sockets, one called the listening socket and the other called the completed connection socket.

Once the connection is successfully established, both parties begin to read and write data using the read and write functions, just like writing to a file stream.

IP_Base#

IP is responsible for communication transmission between two networks that are "not directly connected";

MAC's role is to facilitate communication between two devices that are "directly connected."

Classification of IP Addresses#

Classified addresses: Class A, Class B, Class C, Class D, Class E.

Classless addresses CIDR: A 32-bit IP address is divided into two parts, with the front being the network number and the back being the host number.

Classless Inter-Domain Routing (CIDR)

Subnet mask, meaning to mask the host number, leaving the network number. By performing a bitwise AND operation between the subnet mask and the IP address, the network number can be obtained.

The subnet mask can be used to divide the network number and host number, and it also serves the purpose of subnetting.

Subnetting essentially divides the host address into two parts: subnet network address and subnet host address.

IP Address and Routing Control#

The network address part of an IP address is used for routing control.

The routing control table records the network address and the next address to which it should send packets to the router. Both hosts and routers have their own routing control tables.

When sending an IP packet, the target address in the IP packet header must first be determined, and then the routing control table is checked for records with the same network address. Based on that record, the IP packet is forwarded to the corresponding next router. If there are multiple records with the same network address in the routing control table, the one with the longest match (most bits in common) is chosen.

The loopback address is a default address used for network communication between programs on the same computer.

Computers use a special IP address 127.0.0.1 as the loopback address. A hostname called localhost has the same meaning as this address. When using this IP or hostname, the data packet does not flow to the network.

IP Fragmentation and Reassembly#

The maximum transmission unit (MTU) of each data link is different; for example, the MTU for FDDI data links is 4352, while the MTU for Ethernet is 1500 bytes.

The reason for the different MTUs of each data link is that each type of data link serves different purposes. Different purposes allow for different MTUs.

When the size of an IP packet exceeds the MTU, the IP packet will be fragmented.

Once fragmented, the reassembly of the IP datagram can only be performed by the destination host; routers do not perform reassembly.

In fragmented transmission, if any fragment is lost, the entire IP datagram becomes invalid. Therefore, TCP introduces MSS, meaning fragmentation occurs at the TCP layer rather than the IP layer. For UDP, we should avoid sending a datagram larger than MTU.

Basic Understanding of IPv6#

IPv4 addresses are 32 bits long (4 bytes), represented in dotted-decimal format with each 8 bits as a group.

IPv6 addresses are 128 bits long (16 bytes), represented with each 16 bits as a group, separated by colons ":".

If there are consecutive zeros, these zeros can be omitted and replaced with two colons "::". However, an IP address can only contain two consecutive colons once.

IPv6 not only increases the number of allocable addresses but also has many highlights.

• IPv6 can be automatically configured, allowing automatic IP address assignment even without a DHCP server, making it plug-and-play.
• The header length of IPv6 is fixed at 40 bytes, removing the header checksum, simplifying the header structure, reducing the load on routers, and greatly improving transmission performance.
• IPv6 has network security features to combat IP address forgery and prevent line eavesdropping, significantly enhancing security.
• ...

Improvements in IPv6 headers compared to IPv4:

• The header checksum field has been removed. Since checks are performed at both the data link and transport layers, IPv6 directly eliminates the IP checksum.
• Fragmentation/reassembly-related fields have been removed. Fragmentation and reassembly are time-consuming processes; IPv6 does not allow fragmentation and reassembly at intermediate routers; this operation can only occur at the source and destination hosts, greatly improving the speed of router forwarding.
• The options field has been removed. The options field is no longer part of the standard IP header but may appear at the location indicated by the "next header" in the IPv6 header. Removing the options field makes the IPv6 header a fixed length of 40 bytes.

IP Protocol Related Technologies#

DNS Domain Name Resolution#

DNS can automatically convert domain names into specific IP addresses.

The hierarchical relationship of domain names resembles a tree structure:

• Root DNS server
• Top-level domain DNS server (com)
• Authoritative DNS server (server.com)

The browser first checks its cache; if not found, it requests the operating system's cache. If still not found, it checks the local domain name resolution file hosts. If still not found, it queries the DNS server, with the process as follows:

1. The client first sends a DNS request asking for the IP of www.server.com to the local DNS server (the DNS server address specified in the client's TCP/IP settings).
2. If the local DNS server finds the IP address of www.server.com in its cache, it directly returns it. If not, the local DNS will ask its root DNS server: "Can you tell me the IP address of www.server.com?". The root DNS server is the highest level; it does not directly resolve domain names but can point the way.
3. The root DNS receives the request from the local DNS and finds that the suffix is .com, saying: "www.server.com is managed by the .com zone; I will give you the address of the .com top-level domain server; you can ask it."
4. The local DNS receives the address of the top-level domain server and sends a request asking, "Can you tell me the IP address of www.server.com?"
5. The top-level domain server says: "I will give you the address of the authoritative DNS server responsible for www.server.com; you can ask it."
6. The local DNS then queries the authoritative DNS server: "Can you tell me the IP address of www.server.com?" The authoritative DNS server for server.com is the original source of the domain name resolution result. It is called authoritative because it is in charge of its domain name.
7. The authoritative DNS server queries and tells the local DNS the corresponding IP address X.X.X.X.
8. The local DNS then returns the IP address to the client, allowing the client to establish a connection with the target.

The process of DNS domain name resolution is similar to asking for directions in daily life, only pointing the way without leading.

ARP Address Resolution Protocol#

ARP is a network transmission protocol that finds data link layer addresses by resolving network layer addresses.

ARP determines MAC addresses using ARP requests and ARP responses.

• The host broadcasts an ARP request, which contains the IP address of the host whose MAC address is being sought.
• When all devices on the same link receive the ARP request, they unpack the contents of the ARP request packet. If the target IP address in the ARP request matches their own IP address, that device will place its MAC address into an ARP response packet and return it to the host.

Operating systems typically cache the MAC addresses obtained through ARP for the first time to facilitate direct retrieval of the corresponding MAC address for a given IP address in the future.

The ARP protocol seeks a MAC address given an IP address, while the RARP protocol does the opposite, seeking an IP address given a MAC address.

Additionally, when the sending host and destination host are not on the same local area network, even if the MAC address is known, they cannot communicate directly; they must go through a router for IP layer forwarding, as the router isolates the link layer of the local area network (unless explicit forwarding at the network layer is performed, the router will not automatically forward Ethernet frames from one local area network to another or to the external network. If it could, imagine how terrifying it would be; invalid broadcast frames could flood the entire network, causing network paralysis, which is one of the basic functions of a router: to isolate networks). Therefore, in this case, the sending host will use a gateway IP address as the destination IP address (this is determined at the IP layer), and the MAC address obtained through the ARP protocol will not be the real MAC address of the destination host but rather the MAC address of a router that can reach outside the local area network. Subsequently, all frames sent by the sending host to the destination host will be sent to that router, which will forward them externally. This situation is referred to as delegated ARP or ARP proxy.

DHCP Dynamic Host Configuration Protocol#

DHCP is a communication protocol that enables network administrators to centrally manage and automatically allocate IP network addresses.

The DHCP client process listens on port 68, while the DHCP server process listens on port 67.

• The client first initiates a DHCP discovery message (DHCP DISCOVER) IP datagram. Since the client does not have an IP address and does not know the address of the DHCP server, it uses UDP broadcast communication, with the broadcast destination address being 255.255.255.255 (port 67) and using 0.0.0.0 (port 68) as the source IP address. The DHCP client passes this IP datagram to the link layer, which then broadcasts the frame to all devices on the network.
• When the DHCP server receives the DHCP discovery message, it responds to the client with a DHCP offer message (DHCP OFFER). This message also uses the IP broadcast address 255.255.255.255 and carries information about the server's offered leaseable IP address, subnet mask, default gateway, DNS server, and IP address lease duration.
• After the client receives one or more DHCP offer messages from servers, it selects one server and sends a DHCP request message (DHCP REQUEST) in response, echoing the configured parameters.
• Finally, the server responds to the DHCP request message with a DHCP ACK message, confirming the requested parameters.

Once the client receives the DHCP ACK, the interaction is complete, and the client can use the IP address allocated by the DHCP server during the lease period.

If the leased DHCP IP address is about to expire, the client will send a DHCP request message to the server:

• If the server agrees to continue the lease, it will respond with a DHCP ACK message, and the client will extend the lease.
• If the server does not agree to continue the lease, it will respond with a DHCP NACK message, and the client must stop using the leased IP address.

It can be observed that the entire DHCP interaction uses UDP broadcast communication.

With the introduction of DHCP relay agents, IP address allocation across different subnets can also be centrally managed by a single DHCP server.

• The DHCP client sends a DHCP request packet to the DHCP relay agent, which, upon receiving this broadcast packet, forwards it to the DHCP server in unicast form.
• The server receives this packet and responds to the DHCP relay agent, which then broadcasts this packet to the DHCP client.

Thus, the DHCP server can achieve unified allocation and management of IP addresses even if it is not on the same link.

NAT Network Address Translation#

Network Address Translation (NAT) is a technique that rewrites the source or destination IP address or port of IP packets as they pass through a router or firewall. This technique is commonly used in private networks with multiple hosts that access the internet through a single public IP address.

In simple terms, NAT translates the private IP addresses of hosts within a company, home, or classroom into a public IP address for external communication.

NAT can also translate the combination of IP address + port number. This way, a single global IP address can be used, and this translation technique is called Network Address and Port Translation (NAPT).

The NAPT router's translation table can accurately translate the combination of addresses and ports.

Since both NAT and NAPT rely on their own translation tables, the following issues may arise:

• External hosts cannot actively connect to NAT internal servers because the NAPT translation table does not have a translation record.
• The generation of translation tables and the translation operations incur performance overhead.
• If the NAT router restarts during communication, all TCP connections will be reset.

The solutions mainly involve two approaches.

• The first is to switch to IPv6.

• NAT traversal technology

  In NAT traversal technology, applications behind NAT devices take the initiative; they are aware that the NAT device will modify their outgoing packets, so they actively cooperate with the NAT device's operations to establish mappings, rather than relying on the NAT device to create mappings.

  The client actively obtains a public IP address from the NAT device, then establishes port mapping entries, and uses this entry for external communication, eliminating the need for the NAT device to perform translations.

ICMP#

ICMP stands for Internet Control Message Protocol.

ICMP primarily serves the functions of confirming whether IP packets successfully reach the target address, reporting reasons for discarded IP packets during transmission, and improving network settings.

ICMP can be broadly divided into two categories:

• One category is diagnostic query messages, known as "query message types."
• The other category is error messages that notify of error reasons, known as "error message types."

IGMP#

IGMP is the Internet Group Management Protocol, operating between hosts (multicast members) and the last-hop router.

• IGMP messages request routers to join and leave multicast groups. By default, routers do not forward multicast packets to connected hosts unless the hosts join the multicast group via IGMP. When a host requests to join a multicast group, the router records this in the IGMP router table, and subsequently forwards multicast packets to the corresponding hosts.
• IGMP messages are encapsulated in IP, with the protocol number in the IP header set to 2, and the TTL field value is usually set to 1 because IGMP operates between the host and the connected router.

Multicast addresses are not used for machine IP addresses, as multicast addresses do not have network numbers and host numbers, so they are unrelated to DHCP. Multicast addresses are generally used for the UDP protocol; when a machine sends UDP multicast data, the target address is filled with the multicast address, allowing all machines within the multicast group to receive the data packets.

Joining or leaving a multicast group is implemented through a socket interface, and the host IP does not need to change.

Others#

• OSPF (Open Shortest Path First): An interior gateway protocol (IGP) widely used as a dynamic routing protocol, based on link-state algorithms, considering factors such as link bandwidth and latency to select the best path.
• RIP (Routing Information Protocol): An interior gateway protocol (IGP) that is also a dynamic routing protocol, based on distance-vector algorithms, using fixed hop counts as a metric to select the path with the fewest hops as the best path.
• BGP (Border Gateway Protocol): A routing protocol used to exchange network layer reachability information (NLRI) between routing domains, offering high flexibility and scalability.