HTTPRespond to common status codes

Blog link:Aspects of Network Knowledge

Status code	describe
100-199	Successful receipt of the request requires the client to continue submitting the next request to complete the entire process.
200-299	Successfully received the request and completed the whole processing process. Commonly used200
300-399	In order to complete the request, the requirements need to be further refined: for example, the requested resource has moved to a new address, which is commonly used302(Redirection),307And304(Take cache)
400-499	The client’s request has errors, including syntax errors or cannot be executed correctly. Common404(The requested resources arewebNot in the server),403(Server Denies Access, Insufficient Rights)
500-599	An error occurred on the server side
200	Indicates that everything is normal and returns the normal request result.
302/307	Temporary redirection, indicating that the requested document has been temporarily moved elsewhere, the newurlInlocationGiven in the response header
304	Not modified, indicating that the version of the client cache is up to date and the client should continue to use it.
403	No, the server understands the client request but refuses to process it, which is usually caused by permission setting for files or directories on the server.
404	No, the resource requested by the client does not exist on the server
500	Internal server error, server sidecgi,asp,jspSuch as program error

TCPThree-way handshake and Four Waves

• establishTCPConnection requirementsthree-way handshake: First of allClientThe terminal sends a connection request message,ServerAfter receiving the connection, the terminal repliesACKMessage and allocate resources for this connection.ClientEnd receivedACKAfter the message also toServerEnd-to-end transmissionACKMessage, and allocate resources, so thatTCPThe connection is established.

  • Step 1: Client’sTCPFirst to the serverTCPSend a connection request message. This special message does not contain application layer data; theSYNThe flag bit is set1. In addition, the client will randomly select a starting sequence numberseq=x(The connection request message does not carry data, but consumes a serial number).
  • Step 2: Server-sideTCPAfter receiving the connection request message, if it agrees to establish the connection, it will send the request to the client and send the request to the clientTCPConnection allocationTCPCaches and variables. In the confirmation message,SYNAndACKThe bits are all set to1, confirm the good field value isx+1And the server randomly generates a starting sequence numberseq=y(Confirm that the message does not carry data, but also consumes a serial number). The confirmation message also does not contain application layer data.
  • Step 3: When the client receives the confirmation message, it will also give confirmation to the server and allocate cache and variables to the connection. Of this messageACKThe flag bit is set to1, serial number field isx+1, the confirmation number field isy+1.

• Four waves

  • Step 1: The client intends to close the connection and sends itTCPSending a connection release message, stopping sending data again, and actively closingTCPConnection of this messageFINThe flag bit is set1,seq=uThat is equal to the sequence number of the last byte of data that has been previously transmitted plus1(FINEven if the message does not carry data, it will consume a serial number).
  • Step 2: The server will send out an acknowledgement after receiving the connection release message. The acknowledgement number isack=u+1, the message’s own serial number isvThat is equal to the last own sequence number of the data it has previously transmitted plus1. At this point, the connection from the client to the server is released.TCPThe connection is semi-closed. However, if the server sends data, the client still needs to receive it, that is, the connection from the server to the client is still not closed.
  • Step 3: Notify if the server no longer has data to send to the clientTCPRelease the connection, at which point it issuesFIN=1The connection release message of.
  • Step 4: After receiving the connection release message, the client must issue a confirmation. In the confirmation message,ACKThe field is set to1, confirmation numberack=w+1, serial numberseq=u+1. At this time,TCPThe connection has not been released, and the waiting time set by the timer must pass.2MSLAfter that,ABefore entering the connection closed state.

Computer network architecture

application layer

Application layer (application-layer) is to complete a specific network application through interaction between application processes. Application layer protocols define the rules of communication and interaction between application processes (processes: programs running in the host). Different application layer protocols are required for different network applications. There are many application layer protocols in the Internet, such as domain name system.DNS, which supports World Wide Web applicationsHTTPProtocol that supports e-mailSMTPAgreements, etc. We call the data units interacting with the application layer messages.

Domain name system

Domain Name System (Domain Name SystemAbbreviationsDNS,Domain NameDomain name is one of the core services of the internet. it can be used to connect domain names withIPA distributed database with addresses mapped to each other can make it easier for people to access the Internet without remembering what can be directly read by machines.IPNumber string.

httpProtocol

Hypertext Transfer Protocol (HTTP,HyperText Transfer Protocol) is one of the most widely used network protocols on the Internet. AllWWW(World Wide Web) Documents must comply with this standard.

transport layer

Transport layertransport layer) is responsible for providing common data transmission services to the communication between two host processes. The application process uses this service to transmit application layer messages. “Universal” means that it is not aimed at a specific network application, but multiple applications can use the same transport layer service. Since a host can run multiple threads at the same time, the transport layer has multiplexing and demultiplexing functions. Reuse means that multiple application-layer processes can use the services of the lower transport layer at the same time. In contrast to reuse, reuse means that the transport layer delivers the received information to the corresponding processes in the upper application layer respectively.

Two Protocols Commonly Used in Transport LayerTCP UDP

• Transmission control protocolTCP(Transmisson Control Protocol)-Provide connection-oriented and reliable data transmission services.
• User data protocolUDP(User Datagram Protocol)-Provide connectionless, best-effort data transmission services (no guarantee of data transmission reliability).

TCPThe main characteristics of

• TCPIt is connection-oriented. (Just like making a phone call, you need to dial to establish a connection before the call, and hang up to release the connection after the call is over);
• Every articleTCPA connection can have only two endpoints, eachTCPConnections can only be point-to-point (one-to-one);
• TCPProvide reliable delivery of services. viaTCPThe data transmitted by the connection is error-free, lost, duplicated and arrived in sequence.
• TCPProvide full duplex communication.TCPAllows application processes of both communication parties to send data at any time.TCPBoth ends of the connection are provided with a sending cache and a receiving cache for temporarily storing data communicated by both parties;
• For byte stream.TCPThe “flow” in (Stream) refers to a sequence of bytes flowing into or out of a process. “byte stream oriented” means that although applications andTCPThe interaction of is one block at a time (of different sizes), butTCPThink of the next data of an application as just a series of unstructured byte streams.

UDPThe main characteristics of

• UDPThere is no connection;
• UDPUse best effort delivery, i.e. reliable delivery is not guaranteed, so the host does not need to maintain a complicated link state (there are many parameters in it);
• UDPIt is message oriented;
• UDPThere is no congestion control, so the network congestion will not reduce the transmission rate of the source host (useful for real-time applications, such as live broadcast, real-time video conference, etc.);
• UDPSupport one-to-one, one-to-many, many-to-one and many-to-many interactive communication;
• UDPThe head of the cost is small, only8Bytes, thanTCPThe20The header of bytes is shorter.

network layer

• The two computers communicating in the computer network may pass through many data links and may also pass through many communication subnets. The task of the network layer is to select appropriate network routing and switching nodes to ensure timely data transmission. When sending data, the network layer encapsulates the message segments or user datagrams generated by the transport layer into packets and transmits them. InTCP/IPIn the architecture, due to the use of network layerIPProtocol, so grouping is also calledIPDatagramdatagram.
• The Internet is composed of a large number of heterogeneous (heterogeneous) The network passes through routers (router) connected to each other. The network layer protocol used by the internet is connectionless internet protocol (Intert Prococol) and many routing protocols, the network layer of the internet is also called the internet layer orIPThe floor.

data link layer

• Data link layer (data link layer) is commonly referred to as link layer for short. The data transmission between two hosts is always carried on the link section by section, which requires the use of special link layer protocols. When data is transferred between two neighboring nodes, the data link layer transfers the nextIPDatagrams are assembled into frames and transmitted over a link between two adjacent nodes. Each frame includes data and necessary control information (such as synchronization information, address information, error control, etc.).
• When receiving data, the control information enables the receiving end to know from which bit a frame starts and to which bit it ends. In this way, after receiving a frame, the data link layer can extract the data part from it and hand it over to the network layer. The control information also enables the receiving end t o detect the presence or absence of errors in the received frame. If an error is found, the data link layer simply discards the frame with the error to avoid wasting network resources by continuing to transmit in the network. If it is ne cessary to correct errors in data transmission at the link layer (that is, the data link layer should not only detect but also correct errors), then the reliability transmission protocol should be adopted to correct the errors. This method will complicate the protocol at the link layer.

physical layer

• The data unit transmitted on the physical layer is bits. Physical layer (physical layer) function is to realize transparent transmission of bit streams between adjacent computer nodes and shield the difference between specific transmission media and physical equipment as much as possible. The data link layer above it does not need to consider what the specific transmission medium of the network is. “Transparent Transmission Bit Stream” means that the bit stream transmitted by the actual circuit has not changed, and this circuit seems invisible to the transmitted bit stream.
• The most important and famous of the various associations used in the Internet isTCP/IPTwo agreements.

The Seven-Layer Architecture Diagram of Computer Network

HTTPAndHTTPSThe difference between

HTTPThe protocol runs onTCPAbove, plaintext transmission, the client and server cannot verify each other’s identity;HTTPSIs wearingSSL(Secure Socket Layer) EnclosedHTTP, run onSSLIn fact,SSLRun onTCPAbove, encryption and authentication mechanisms have been addedHTTP. There are the following differences between the two:

• Different ports:HTTPSAndHTTPUse different connection methods and use different ports. The former is80The latter is443;
• Resource consumption: andHTTPCompared with communication,HTTPSCommunication will consume more due to encryption and decryptionCPUAnd memory resources;
• Expenses:HTTPSCommunications require certificates, which generally need to be purchased from an authentication institution;
• HTTPSThe encryption mechanism of is a hybrid encryption mechanism of shared key encryption and public key encryption.

Symmetric Encryption and Asymmetric Encryption

• Symmetric key encryption refers to the way in which the same key is used for encryption and decryption. The biggest problem in this way is the key send ing problem, i.e. how to safely send the key to the other party. Asymmetric encryption refers to the use of a pair of asymmetric keys, i.e. a public key and a private key. The public key can be released at will, but the private key is only known by itself. The party sending the ciphertext uses the other party’s public key for encryption, and the other party uses its own private key for decryption after receiving the encrypted information.
• Since asymmetric encryption does not require sending the private key used for decryption, security can be ensured; However, compared with symmetric encryption, it is very slow, so we still need to use symmetric encryption to transmit messages, but the key used by symmetric encryption can be sent out through asymmetric encryption.

TCPHow does the protocol maintain the reliability of transmission

TCPThe invention provides a connection-oriented and reliable byte stream service. Where connection-oriented means two usesTCPAn application of (usually a client and a server) must first establish one before exchanging data with each other.TCPConnection. In oneTCPIn the connection, only two parties communicate with each other; While byte stream service means that two applications pass throughTCPLink exchange8bitByte stream consisting of bytes,TCPRecord identifiers are not inserted into byte streams.

For reliability,TCPGuaranteed by:

• Packet check: The purpose is to detect any change of data during transmission. If the packet is found to be wrong, the packet segment is discarded and no response is given. At this timeTCPThe sending end will retransmit the data after timeout.
• Reorder out-of-order packetsSinceTCPSegment asIPDatagram to transmit, andIPThe arrival of datagrams may be out of order, thereforeTCPThe arrival of message segments may also be out of order.TCPReorder the out-of-order data before handing it over to the application layer.
• Discard duplicate data: For duplicate data, duplicate data can be discarded;
• Response mechanism: whenTCPReceived fromTCPConnect the data at the other end and it will send an acknowledgement. This confirmation is not sent immediately and will usually be delayed by a fraction of a second.
• Timeout retransmission: whenTCPAfter sending a segment, it starts a timer and waits for the destination to acknowledge receipt of the segment. If a confirmation cannot be received in time, the message segment will be retransmitted.
• Flow control：TCPEach side of the connection has a fixed size buffer space.TCPThe receiving end of only allows the other end to send the data that the receiving end buffer can accept, which can prevent the faster host from causing the buffer of the slower host to overflow, which is flow control.TCPThe flow control protocol used is a variable size sliding window protocol.

Find the corresponding domain nameIPAddress

This step includesDNSThe specific search process includes: browser cache-> system cache-> router cache …

• The browser searches for its ownDNSCache (maintain a domain name andIPAddress correspondence table);
• Search forDNSCache (maintain a domain name andIPAddress correspondence table);
• Search for operating systemhostsFile (WindowsUnder the environment, maintain a domain name andIPAddress correspondence table);

• The operating system sends the domain name toLDNS(Local Domain Name Server),LDNSInquire about one’s ownDNSCaching (general search success rate is80%About), search success will return the result, failure will initiate an iterationDNSResolve request:

  • LDNSToRoot Name Server(root domain name server, such ascom、net、orgE.g., the address of the resolved top-level domain name server, where,Root Name ServerReturncomThe address of the domain’s top-level domain name server;
  • LDNSTocomThe top-level domain name server of the domain initiates the request and returnsbaidu.comDomain name server address;
  • LDNSTobaidu.comThe domain name server initiates a request and obtainswww.baidu.comTheIPAddress;
• LDNSWill getIPThe address is returned to the operating system, and it will alsoIPAddress cache;
• The operating system willIPThe address is returned to the browser, and at the same time it willIPThe address is cached.

From inputURLWhat Happens to Page Loading

Generally speaking, it is divided into the following processes:

• DNSanalysis
• TCPConnection
• SendHTTPRequest
• The server processes the request and returnsHTTPMessage
• Browser Parses Rendered Page
• End of connection

HTTPThe purpose of several request methods of

• GETMethod: Send a request to get a resource on the server
• POSTMethods: ToURLThe specified resource submits data or appends new data.
• PUTMethods: FollowPOSTThe method is similar to submitting data to the server. However, there are differences between them.PUTSpecifies the location of the resource on the server, andPOSTNo
• HEADMethod: Only the header of the page is requested
• DELETEMethod: Delete a resource on the server
• OPTIONSMethod: It is used to get the currentURLSupported methods. If the request is successful, there will be oneAllowThe header of contains something like “GET,POST“Such information
• TRACEMethods:TRACEThe method is used to trigger a remote, application-level request message loop
• CONNECTMethod: Convert the requested connection to transparentTCP/IPChannel

Five categoriesIPRange of addresses

IPThe addresses are divided intoA,B,C,D,EFive categories.

• Network number: used to identify the network where the host is located;
• Host Number: used to identify hosts in the network.

among themAClass is allocated to government agencies for use.BClass addresses are used by large and medium-sized enterprises.CClass address for personal use. These three are the main ones.

IPThere are five types of addresses.AClass is reserved for government agencies.BClass is allocated to medium-sized companies.CClass is assigned to anyone who needs it.DClass is used for multicast.EClasses are used for experiments, and the number of addresses each class can accommodate is different.

among themAClass,BClasses,, andCClass These three types of addresses are used toTCP/IPNodes, other two typesDClasses andEClass is used for special purposes.A、B、CThree categoriesIPCharacteristics of Address: When HandleIPWhen the address is written in binary form,AThe first bit of a class address is always0,BThe first two digits of a class address are always10,CThe first three bits of a class address are always110.

AClass address

1. AClass address1Bytes are network addresses, others3Bytes are host addresses.
2. AClass address range:1.0.0.1—126.155.255.254

3. APrivate and reserved addresses in class addresses:

   • 10.X.X.XIt is a private address (the so-called private address is an address that is not used on the Internet but is used in a local area network)
   • 127.X.X.XIs reserved address, used for loop test

BClass address

1. BClass address1Bytes and2Bytes are network addresses, others2Bytes are host addresses.
2. BClass address range:128.0.0.1—191.255.255.254.

3. BPrivate and reserved addresses for class addresses:

   • 172.16.0.0—172.31.255.255Is a private address
   • 169.254.X.XIt’s a reserved address. If yoursIPThe address is automatically obtainedIPAddress, and you can’t find one on the networkDHCPServer. You will get one of them.IP

CClass address

1. CClass address1Bytes, first2Bytes and3Bytes are network addresses, first4Each byte is the host address. In addition, the first1The first three bits of bytes are fixed as110.
2. CClass address range:192.0.0.1—223.255.255.254.

3. CPrivate address in class address:

   • 192.168.X.XIt’s a private address.

DClass address

1. DClass address does not distinguish between network address and host address, its first1The first four bits of bytes are fixed as1110.
2. DClass address range:224.0.0.1—239.255.255.254

EClass address

1. EClass address also has no distinction between network address and host address, its first1The first five bits of bytes are fixed as11110.
2. EClass address range:240.0.0.1—255.255.255.254

HTTPLong connection, short connection

• InHTTP/1.0Short connections are used by default in. That is to say, each time the client and the server performHTTPOperation, establish a connection, the end of the task will interrupt the connection. When a client browser accesses a certainHTMLOr other types ofWebThe page contains otherWebResources (e.g.JavaScriptDocuments, image files,CSSFile, etc.), every encounter such a.WebResources, the browser will re-create oneHTTPConversation.
• And fromHTTP/1.1Starting from, long connections are used by default to maintain connection characteristics. Using long connectionsHTTPProtocol, will add this line of code in the response header:Connection:keep-alive
• In the case of long connection, when a web page is opened, it is used for transmission between the client and the server.HTTPData-basedTCPThe connection will not be closed, and the client will continue to use the established connection when accessing the server again.Keep-AliveThe connection will not be permanently maintained, it has a retention time that can be used in different server software (e.g.Apache) to set this time. Realizing long connection requires both client and server to support long connection.
• HTTPThe long and short connections of the protocol are essentiallyTCPLong and short connections of the protocol.

How to understandHTTPThe protocol is stateless

HTTPThe protocol is stateless, which means that the protocol has no memory for transaction processing, and the server does not know what state the client is in. In other words, there is no connection between the opening of a web page on a server and the last opening of a web page on this server.HTTPIs a stateless connection-oriented protocol, stateless does not meanHTTPCan’t keepTCPConnection, more can’t representHTTPWhat is used isUDPProtocol (no connection).

Various agreements andHTTPRelationship between agreements

SocketConnection withHTTPConnection and Difference of Connection

• under normal conditionsSocketConnection isTCPConnection, thereforeSocketOnce the connection is established, the communication parties can start sending data content to each other until the connection between the two parties is disconnected. However, in actual network applications, the communication between the client and the server often needs to pass through multiple intermediate nodes, such as routers, gateways, firewalls, etc., and most firewalls close inactive connections for a long time by default, resulting inSocketThe connection is disconnected, so the network needs to be informed by polling that the connection is active.
• AndHTTPThe connection uses “Request-response“mode, not only needs to establish a connection when requesting, but also needs the client to send a request to the server before the server can reply to the data.
• In many cases, it is necessary for the server to actively push data to the client to keep the real-time and synchronous data between the client and the server. At this time, if the two sides set up isSocketConnection, the server can directly transmit data to the client; If the two sides set up isHTTPConnection, the server needs to wait for the client to send a request before returning the data to the client. Therefore, the client sends a connection request to the server regularly, not only to remain online, but also to “ask” the server if there is any new data, and if so, to send the data to the client.

HTTP(TCP) message structure

For example, one100kbTheHTMLThe document needs to be transferred to another computer, instead of directly transferring the entire document, it may be cut into several parts, such as four25kbThe data section of. And each data segment is added with oneTCPThe first part, formedTCPMessage.TCPMessage (Segment), including a header and a data portion.

Heading:

• Source portsource port
• Destination portdestination port
• Serial numbersequence number
• Confirmation numberacknowledgment number
• Data offsetoffset
• reservereserved
• flag bittcp flags
• window sizewindow size
• Inspection andchecksum
• Emergency pointerurgent pointer
• Optionstcp options

HTTPThe caching mechanism of

HTTPThe cache of mainly usesheaderIn the two fields to control:

• Cache-controlIt mainly includes and several fields:

  • private: only the client can cache
  • public: Both client and proxy servers can cache
  • max-age: cache expiration time
  • no-cache: comparison cache is required to verify cached data
  • no-store: All memory will not be cached

• ETag: cache for comparison,EtagIs an identification code for server resources

  • When the client sends the first request, the server will issue the identification code of the currently requested resourceEtagThe next time you request it, the client will passheaderFrom insideIf-None-MatchPut this identification codeEtagTake it with you. The server will send it from the clientEtagWith the latest resourcesEtagFor comparison, if the same, it means that the resource has not been updated and returns304.

viaCache-controlAndEtagTo achieve with the cooperation ofHTTPThe cache mechanism of.

Cookie

CookieIs used to remember some state in the local cache, oneCookieGenerally includesdomain(domain),path、Expires(Expiration Time) and other attributes. The server can send a message to the server through the response headerset-cookiesWrites the state to the client’sCookieChina.

HTTP 2.0AndHTTP 1.xWhat are the advantages compared with

• binary format：HTTP 1.xIs a text protocol, andHTTP 2.0It is a binary system with frame as the basic unit. It is a binary protocol. A frame contains not only data but also the identification of the frame:Stream IdentifierThat is, which frame the frame belongs to is identifiedrequest, making network transmission very flexible.

• MultiplexingA great improvement, originallyHTTP 1.xThe situation of connecting one request at a time has great limitations, and also causes many problems, such as the consumption and efficiency of establishing multiple connections.

  • HTTP 1.xIn order to solve the problem of efficiency, it is possible to launch as many concurrent requests as possible to load resources. However, browsers have restrictions on concurrent requests under the same domain name, and the optimization method is generally to put the requested resources under different domain names to break through this restriction.
  • AndHTTP 2.0Supported multiplexing can solve this problem very well. Multiple requests share oneTCPConnection, multiple requests can be in this at the same timeTCPConnection concurrency, one is to solve the establishment of multipleTCPOne of the problems of connection consumption also solved the problem of efficiency. So what principle supports multiple requests that can be made in oneTCPWhat about concurrency on the connection? The basic principle is the above binary framing, because each frame has an identity, so different frames of multiple requests can be sent out out of order concurrently, and the server will sort them into corresponding ones according to the identity of each frame.requestChina.
• headerHead compression: mainly through compressionheaderTo reduce the size of the request, reduce traffic consumption and improve efficiency. Because there was a problem before, every request must be brought with it.headerAnd thisheaderThe data in is usually the same.
• Support server push

Flow control

Flow control is to control the flow on a communication path, that is, the sender dynamically adjusts the sending rate by obtaining feedback from the receiver to achieve the effect of flow control. Its purpose is to ensure that the sending speed of the sender does not exceed the receiving speed of the receiver.

Congestion control

Congestion control is to control the traffic of the whole communication subnet and belongs to global control.

1. Slow Start+Congestion Avoidance

2. Fast retransmission+fast recovery

   • Fast retransmission: The retransmission mechanism does not start retransmission until the receiver’s reply has not been received after timeout. The design idea of fast retransmission is: if the sender receives3Of duplicate recipientsACK, it can be judged that there is a message segment missing, and the lost message segment can be retransmitted immediately at this time without waiting for the set timeout time to start retransmission, thus improving the retransmission efficiency.
   • Quick recovery: congestion control reduces the congestion window to when the network is congested1, slow start again, so there is a problem is that the network cannot quickly return to normal state. Fast recovery is to optimize this problem, using fast recovery, when congestion occurs, the congestion window will only be reduced to the new slow start gate value (i.e.12), and will not drop to1, and then directly begin to enter congestion to avoid additive growth.

Original link:A Complete Book of Network Knowledge Points Applicable to Both Front and Back Ends