In-depth understanding of browser caching mechanism

  css, html, html5, javascript, node.js

I introduction

Caching can be said to be a simple and efficient optimization method in performance optimization. An excellent caching strategy can shorten the distance for web pages to request resources and reduce the delay. Moreover, since cached files can be reused, it can also reduce bandwidth and network load.

For a data request, it can be divided into three steps: initiating a network request, back-end processing, and browser response. Browser caching can help us optimize performance in the first and third steps. For example, if the cache is directly used without initiating the request, or if the request is initiated but the data stored at the back end is consistent with that of the front end, there is no need to return the data back, thus reducing the response data.

In the following content, we will discuss the browser caching mechanism through cache location, cache strategy and actual scenario application cache strategy.

If you need to get a mind map or want to read more excellent articles, please stamp it fiercely.GitHub blog

Second, the cache location

There are four types of cache locations and each has its own priority. When the cache is searched in sequence and there are no hits, the network will be requested.

  • Service Worker
  • Memory Cache
  • Disk Cache
  • Push Cache

1.Service Worker

Service Worker is a separate thread running behind the browser and can be used to implement caching functions. With Service Worker, the transport protocol must be HTTPS. Because request interception is involved in Service Worker, HTTPS protocol must be used to ensure security.Service Worker’s cache is different from other built-in caching mechanisms in browsers. It allows us to freely control which files to cache, how to match the cache, how to read the cache, and the cache is persistent..

The realization of caching function by Service Worker is generally divided into three steps: first, Service Worker needs to be registered, and then the required files can be cached after listening to the install event. then the next time the user accesses, whether there is cache can be queried by intercepting the request. if there is cache, the cached files can be read directly, otherwise, the data can be requested.

When the Service Worker fails to hit the cache, we need to call the fetch function to get the data. In other words, if we do not hit the cache in Service Worker, we will look up the data according to the cache search priority. However, whether we get the data from the Memory Cache or the network request, the browser will display the content we get from the Service Worker.

2.Memory Cache

Memory Cache is also the cache in memory, which mainly contains the resources that have been captured in the current page, such as styles, scripts, pictures, etc. that have been downloaded on the page. Reading the data in the memory is definitely faster than the disk. Although the memory cache reads efficiently, the cache duration is very short and will be released as the process is released.Once we close the Tab page, the cache in memory is released.

So since the memory cache is so efficient, can we keep all the data in memory?
It is out of the question. The memory in the computer must be much smaller than the capacity of the hard disk. The operating system needs to carefully calculate the use of memory, so the memory that can be used by us must not be much.

When we refresh the page again after visiting the page, we can find that a lot of data comes from the memory cache.

An important cache resource in the memory cache is a preloader-related instruction (for example<link rel="prefetch">) downloaded resources. It is well known that preloader’s related instructions are already one of the common methods of page optimization. It can parse js/css files while the network requests the next resource.

What needs attention is that,When caching resources, the memory cache does not care about the value of HTTP Cache-Control, which returns the resources. At the same time, the matching of resources is not only to match URL, but also to check other characteristics such as Content-Type, CORS, etc..

3.Disk Cache

Disk Cache is also the cache stored in the hard disk. The reading speed is slower, but everything can be stored in the disk.Compared with Memory Cache, it is better in capacity and storage timeliness..

Disk Cache coverage is the largest of all browser caches. It will determine which resources need to be cached, which resources can be used directly without request, and which resources have expired and need to be requested again according to the fields in the HTTP Herder. And even in the case of cross-site, once resources with the same address are cached by the hard disk, they will not request data again. Most of the cache comes from Disk Cache. We will describe the cache fields in the protocol header of HTTP in detail below.

Which files will the browser drop into memory? Which are thrown into the hard disk?
There are different opinions on this point online, but the following opinions are more reliable:

  • For large files, the large probability is not stored in memory, and vice versa
  • If the current system memory usage rate is high, the files will be preferentially stored in the hard disk.

4.Push Cache

Push Cache (Push Cache) is the content in HTTP/2. It will only be used when none of the above three caches has hit.It only exists in the Session and is released once the session ends, and the cache time is also very short.In Chrome browser, it only takes about 5 minutes. At the same time, it does not strictly execute cache instructions in HTTP headers.

Push Cache can find very little information in China, also because HTTP/2 is not popular enough in China. It is recommended to read here.Jake ArchibaldTheHTTP/2 push is tougher than I thoughtThis article, several conclusions in the article:

  • All resources can be pushed and cached, but Edge and Safari browsers have relatively poor support.
  • Resources that can push no-cache and no-store
  • Once the connection is closed, Push Cache is released
  • Multiple pages can use the same HTTP/ 2 connection or the same Push Cache. This mainly depends on the implementation of browsers. For performance reasons, some browsers will use the same HTTP connection for the same domain name but different tab tags.
  • The cache in Push Cache can only be used once.
  • The browser can refuse to accept the existing push of resources.
  • You can push resources to other domain names

If none of the above four caches hit, then only a request can be initiated to obtain resources.

Then for the sake of performance, most interfaces should choose a good caching strategy.Generally, browser caching policies are divided into two types: strong caching and negotiation caching, and caching policies are implemented by setting HTTP Header.

Three, cache process analysis

The browser communicates with the server in response mode, that is, the browser initiates an HTTP request-the server responds to the request.So how does the browser determine whether a resource should be cached and how to cache it?? After the browser receives the request result after initiating the request to the server for the first time, the browser stores the request result and the cache identifier into the browser cache.The browser’s handling of the cache is determined by the response header returned when the resource is first requested.. The specific process is as follows:

第一次发起HTTP请求

From the above figure, we can know:

  • Every time the browser initiates a request, it will first look up the result of the request and the cache identifier in the browser cache.
  • Every time the browser gets the returned request result, it will store the result and the cache identifier in the browser cache.

The above two conclusions are the key to the browser caching mechanism, which ensures the caching and reading of each request. As long as we understand the usage rules of browser caching again, all problems will be solved. This article will also make a detailed analysis around this point. In order to make it easier for everyone to understand, here we divide the caching process into two parts according to whether we need to reissue HTTP requests to the server, namely, strong caching and negotiation caching.

Four, strong cache

Strong cache: it will not send a request to the server and directly read the resources from the cache. You can see the status code of 200 returned by t he request in the Network option of chrome console, and the Size displays from disk cache or from memory cache. Strong caching can be achieved by setting two HTTP Header: Expires and Cache-Control.

1.Expires

Cache Expiration Time, used to specify the expiration time of resources, is a specific point in time on the server side. In other words, Expires=max-age+request time, which needs to be used in combination with Last-modified. Expires is the header field of the Web server response message. When responding to http requests, it tells the browser that the browser can fetch data directly from the browser cache before the expiration time without requesting again.

Expires is a product of HTTP/1, which is limited by local time. If the local time is modified, cache invalidation may occur..Expires: Wed, 22 Oct 2018 08:41:00 GMTIndicates that the resource will expire after Wed, 22 Oct 2018 08:41:00 GMT and needs to be requested again.

2.Cache-Control

In HTTP/1.1, Cache-Control is the most important rule, which is mainly used to control the cache of web pages. For example, whenCache-Control:max-age=300, it means that the resource will be loaded again within 5 minutes of the correct return time of the request (which will be recorded by the browser) and the strong cache will be hit.

Cache-Control can be set in the request header or the response header, and various instructions can be used in combination:

publicAll content will be cached (both client and proxy servers can cache). Specifically, the response can be cached by any intermediate node, such as Browser <-proxy1 <-proxy2 <-server. the intermediate proxy can cache resources, such as requesting the same resource proxy1 to directly give what it caches to browser next time instead of asking proxy2.

privateAll content can only be cached by the client, Cache-Control default value. Specifically, it means that intermediate nodes are not allowed to cache. for browser <-proxy1 <-proxy2 <-Server, the proxy will honestly send the data returned by the server to proxy1 and will not cache any data itself. When Browser requests again next time, proxy will do a good job of forwarding the request instead of self-assertion of cached data for itself.

no-cache: The client caches the contents, and whether to use the cache needs to be verified through negotiation cache. Indicates that the cache control method of Cache-Control is not used for pre-verification, but Etag or Last-Modified field is used to control cache.It should be noted that the name no-cache is somewhat misleading. After setting no-cache, it does not mean that the browser will no longer cache data, but when using cached data, the browser needs to confirm whether the data is still consistent with the server.

no-store: All content will not be cached, i.e. neither forced cache nor negotiation cache will be used

max-age: max-age=xxx (xxx is numeric) indicates that the cache contents will expire in xxx seconds.

s-maxage(Unit S): Same as max-age, it only takes effect in proxy server (such as CDN cache). For example, when s-maxage=60, even if the CDN content is updated, the browser will not make a request during the 60 seconds. Max-age is used for normal caching, while s-maxage is used for proxy caching.S-maxage takes precedence over max-age.. If s-maxage exists, max-age and Expires header will be overwritten.

max-stale: Maximum tolerable expiration time. The max-stale command indicates that the client is willing to receive a response that has expired. If a value of max-stale is specified, the maximum tolerance time is the corresponding number of seconds. If not specified, then the browser is willing to receive any age response (age indicates the difference between the time the response was generated or confirmed by the source station and the current time).

min-fresh: Minimum freshness that can be tolerated. Min-fresh indicates that the client is unwilling to accept a response with freshness no more than the sum of the current age plus the time set by min-fresh.
cache-control

As we can see from the figure, we can use multiple instructions together to achieve multiple purposes. For example, we hope that the resources can be cached, and both the client and the proxy server can cache, and the cache expiration time can be set, etc.

3. Comparison between 3.Expires and Cache-Control

In fact, there is little difference between the two. The difference is that Expires is the product of http1.0 and Cache-Control is the product of http1.1.If both exist at the same time, Cache-Control priority is higher than Expires.; Expires is useful in some environments that do not support HTTP1.1. Therefore, Expires is actually an outdated product. Its existence at this stage is only a compatible writing style.
The strong cache determines whether to cache or not based on whether it exceeds a certain time or a certain period of time and does not care whether the server-side file has been updated, which may cause the loaded file to be not the latest content of the server-side.How do we know if the server-side content has been updated? At this point we need to use the negotiation cache policy.

V. Negotiation Cache

Negotiation cache is a process in which the browser initiates a request to the server with the cache identifier after the forced cache fails, and the server decides whether to use the cache according to the cache identifier. There are mainly the following two situations

  • Negotiation cache takes effect, returning 304 and Not Modified

协商缓存生效

  • Negotiate cache failure, return 200 and request result

协商缓存失效
Negotiation cache can be implemented by setting two HTTP Header: Last-Modified and ETag.
### 1.Last-Modified and If-Modified-Since
When the browser accesses the resource for the first time, the server returns the resource and adds the Last-Modified header to the response header. the value is the last modification time of the resource on the server. the browser caches the file and header after receiving it.

Last-Modified: Fri, 22 Jul 2016 01:47:00 GMT

The next time the browser requests this resource, the browser detects that there is a Last-Modified header, and adds the header If-Modified-Since the value is the value in Last-Modified. If the server receives this resource request again, it will compare the value in the if-modified-sink with the last modification time of this resource in the server. if there is no change, it will return 304 and the empty response body and read directly from the cache. if the time of the if-modified-sink is less than the last modification time of this resource in the server, it will indicate that the file has been updated, and then it will return the new resource file and 200

But Last-Modified has some drawbacks:

  • If the cache file is opened locally, even if the file is not modified, it will still cause the Last-Modified to be modified, and the server cannot hit the cache and send the same resource.
  • Because Last-Modified can only be timed in seconds, if the file is modified within an imperceptible time, the server will think that the resource still hit and will not return the correct resource.

Since it is still insufficient to decide whether to cache according to the file modification time, can cache policy be decided directly according to whether the file content is modified? So it appeared in HTTP/1.1ETagAndIf-None-Match

2.ETag and If-None-Match

Etag is a unique identifier (generated by the server) that returns the current resource file when the server responds to the request. Etag will be regenerated whenever the resource changes.. When the browser sends a request to the server for the next loading resource, it will put the Etag value returned last time into the If-None-Match in the request header. the server can judge whether the resource has been modified relative to the client by comparing the If-None-Match sent from the client with the ETag of the resource on its server. If the server finds that the ETag does not match, it will directly send the new resource (including the new ETag) to the client in the form of conventional GET 200 packets. If the ETag is consistent, it is sufficient to directly return 304 to inform the client to directly use the local cache.

ETag和If-None-Match

3. Comparison between the two:

  • First, Etag is better than Last-Modified in accuracy.

The time unit of Last-Modified is seconds. If a file changes many times within one second, their Last-Modified does not actually reflect the changes, but Etag changes every time to ensure the accuracy. If it is a load-balanced server, the Last-Modified generated by each server may also be inconsistent.

  • Second, Etag is inferior to Last-Modified in performance. After all, Last-Modified only needs to record time, while Etag needs the server to calculate a hash value through an algorithm.
  • Third, in terms of priority, Etag is given priority in server verification.

VI. Cache Mechanism

The forced cache takes precedence over the negotiation cache. If the forced cache (Expires and Cache-Control) takes effect, the cache is directly used. If it does not take effect, the negotiation cache (Last-Modified/If-Modified-Since and Etag/If-None-Match) is carried out. The negotiation cache is determined by the server whether to use the cache. If the negotiation cache fails, the cache representing the request fails, returns 200, returns the resource and cache identifier again, and then stores it in the browser cache. If it takes effect, it returns 304 and continues to use the cache.. The specific flow chart is as follows:

缓存的机制

See here, I don’t know if you have such a question:If no caching policy is set, what will the browser do?

In this case, the browser will adopt a heuristic algorithm, usually taking Date in the response header minus 10% of the Last-Modified value as the cache time.

Seven, the actual scene application cache strategy

1. Frequently changing resources

Cache-Control: no-cache

For frequently changing resources, first of all need to useCache-Control: no-cacheMake the browser request the server every time, and then cooperate with ETag or Last-Modified to verify whether the resource is valid. Although such an approach cannot save the number of requests, it can significantly reduce the size of response data.

2. Unchanged resources

Cache-Control: max-age=31536000

Usually, when processing such resources, a large Cache-Control is configured for them.max-age=31536000(one year) so that the browser requests the same URL later and hits the mandatory cache. In order to solve the problem of updating, dynamic characters such as hash, version number, etc. need to be added to the file name (or path), and then the dynamic characters need to be changed, so as to achieve the purpose of changing the reference URL and invalidate the previous forced cache (in fact, it is not immediately invalidated, but is no longer used).
Class libraries provided online (e.g.jquery-3.3.1.min.js,lodash.min.jsEtc.) all adopt this mode.

VIII. Influence of User Behavior on Browser Cache

The so-called influence of user behavior on browser caching refers to what caching strategy users will trigger when they operate the browser. There are three main types:

  • Open the webpage and enter the address in the address bar: find out if there is a match in disk cache. If yes, use it. If not, send a network request.
  • Normal refresh (F5): Because TAB is not turned off, memory cache is available and will be used first (if it matches). Disk cache comes next.
  • Forced refresh (Ctrl+F5): the browser does not use cache, so all requests sent have headers withCache-control: no-cache(For compatibility, I also brought it with mePragma: no-cache), the server directly returns 200 and the latest content.

Reference article