This series of articles is《Node.js Design Patterns Second Edition》The translation of the original text and the reading notes were serialized and updated in GitHub.Synchronized translation links.
Welcome to pay attention to my column, and subsequent blog posts will be synchronized in the column:
- Encounter’s Digging Column
- Programming Thinking of Zhihu’s Column Encounter
- Segmentfault front-end station
Node.js Essential Patterns
ForNode.js
Asynchrony is its most prominent feature, but for some other languages, such asPHP
, asynchronous code is not often processed.
In synchronous programming, we are used to imagining the execution of code as a continuous calculation step from top to bottom. Each operation is blocked, which means that the next operation can be executed only after the execution of one operation is completed. This method is beneficial to our understanding and debugging.
However, in asynchronous programming, we can perform some operations such as reading files or executing network requests in the background. When we call the asynchronous operation method, even if the current or previous operation has not been completed, the following subsequent operations will continue to be executed. The operations executed in the background will be completed at any time, and the application will react in the correct way when the asynchronous call is completed.
Although this non-blocking method has better performance than blocking method, it is really difficult for programmers to understand, and asynchronous sequence may become difficult to operate when processing advanced applications with more complex asynchronous control flow.
Node.js
A series of tools and design patterns are provided so that we can best handle asynchronous code. It is very important to know how to use them to write applications that have performance and are easy to understand and debug.
In this chapter, we will see the two most important asynchronous modes: callbacks and event publishers.
Callback mode
As mentioned in the previous chapter, the callback isReactor mode
Thehandler
The callback was originallyNode.js
One of the unique programming styles. Callback functions are functions that propagate the results of asynchronous operations after they are completed. They are always used to replace the return instructions of synchronous operations. AndJavaScript
It happens to be the best language for callback. InJavaScript
In, a function is a first-class citizen. We can pass a function variable as a parameter, call it in another function, and store the result of the call in a certain data structure. Another ideal structure to implement callbacks is closures. Using closures, we can preserve the context in which the function was created, so that whenever a callback is called, the context in which asynchronous operations are requested is maintained.
In this section, we analyze callback-based programming ideas and patterns, rather than patterns of return instructions for synchronous operations.
CPS
InJavaScript
In, the callback function is passed as a parameter to another function and called when the operation is completed. In functional programming, this method of transferring results is calledCPS
. This is a general concept and not only for asynchronous operations. In fact, it just passes the result as a parameter to another function (callback function), and then calls the callback function in the body logic to get the operation result, instead of directly returning it to the caller.
CPS synchronization
For a clearer understandingCPS
, let’s look at this simple synchronization function:
function add(a, b) {
return a + b;
}
The above example has become a direct programming style. In fact, there is nothing special about it. It is the use ofreturn
Statement passes the result directly to the caller. It represents the most common way to return results in synchronous programming. Of the above functionsCPS
Write it as follows:
function add(a, b, callback) {
callback(a + b);
}
add()
The function is synchronizedCPS
Function,CPS
The function only gets it when it is calledadd()
The following code is the calling method for the execution result of the function:
console.log('before');
add(1, 2, result => console.log('Result: ' + result));
console.log('after');
Sinceadd()
Is synchronized, then the above code prints the following results:
before
Result: 3
after
Asynchronous CPS
Then let’s think about the following example, hereadd()
The function is asynchronous:
function additionAsync(a, b, callback) {
setTimeout(() => callback(a + b), 100);
}
In the above code, we usesetTimeout()
Simulates the call of an asynchronous callback function. Now, we calladditionalAsync
And see the specific output results.
console.log('before');
additionAsync(1, 2, result => console.log('Result: ' + result));
console.log('after');
The above code will have the following output:
before
after
Result: 3
Because ..setTimeout()
Is an asynchronous operation, so it will not wait for the callback to be executed, but will return immediately and give control to.addAsync()
And then return it to its caller.Node.js
This attribute in is critical because whenever asynchronous requests are generated, control is given to the event loop, allowing new events from the queue to be processed.
The following picture showsNode.js
In the event cycle process:
When the asynchronous operation is completed, the execution right will be given to the callback function where the asynchronous operation starts. Execution will start fromEvent cycleAt first, so it will have a new stack. ForJavaScript
This is its advantage. It is precisely because closures preserve their context that callbacks can be executed normally even at different points in time and at different locations.
The synchronization function is blocked until it completes the operation. The asynchronous function returns immediately, and the result is passed to the handler (a callback in our example) in a later loop of the event loop.
Non-CPS style callback mode
In some cases, we may think that callback CPS is written asynchronously, but it is not. For example, the following code,Array
Object’smap()
Methods:
const result = [1, 5, 7].map(element => element - 1);
console.log(result); // [0, 4, 6]
In the above example, the callback is only used to iterate over the elements of the array, not to pass the result of the operation. In fact, in this example, the callback method is used to return synchronously instead of passing the result. Is it a callback that passes the result of an operationAPI
The document has clear instructions.
Synchronous or asynchronous?
We have seen that the order of code execution will fundamentally change due to synchronous or asynchronous execution. This has a significant impact on the process, correctness and efficiency of the entire application. The following is an analysis of the two modes and their defects. Generally speaking, it is necessary to avoid confusion that is difficult to detect and expand due to inconsistent execution sequence. The following is an asynchronous instance with traps:
A Problem Function
One of the most dangerous situations is that synchronous execution should be asynchronous execution under certain conditions.API
. Take the following code as an example:
const fs = require('fs');
const cache = {};
function inconsistentRead(filename, callback) {
if (cache[filename]) {
// 如果缓存命中,则同步执行回调
callback(cache[filename]);
} else {
// 未命中,则执行异步非阻塞的I/O操作
fs.readFile(filename, 'utf8', (err, data) => {
cache[filename] = data;
callback(data);
});
}
}
The above functions use cache to store the results of different file read operations. However, remember, this is just an example, it lacks error handling, and its cache logic itself is not optimal (e.g. there is no cache retirement policy). In addition, the above function is very dangerous, because if the cache is not set, its behavior is asynchronous untilfs.readFile()
Until the function returns the result, it will not execute synchronously, and the cache will not trigger, but will go to the asynchronous callback call.
Liberationzalgo
Aboutzalgo
In fact, it refers to the uncertainty of synchronous or asynchronous behavior, which almost always leads to very difficult tracking.bug
.
Now, let’s look at how to use a function whose order is unpredictable. It can even easily interrupt an application. Look at the following code:
function createFileReader(filename) {
const listeners = [];
inconsistentRead(filename, value => {
listeners.forEach(listener => listener(value));
});
return {
onDataReady: listener => listeners.push(listener)
};
}
When the above function is called, it creates a new object that acts as an event publisher, allowing us to set up multiple event listeners for file read operations. When the read operation is completed and the data is available, all listeners will be called immediately. The previous function uses the previously definedinconsistentRead()
Function to implement this function. We are now trying to callcreateFileReader()
Functions:
const reader1 = createFileReader('data.txt');
reader1.onDataReady(data => {
console.log('First call data: ' + data);
// 之后再次通过fs读取同一个文件
const reader2 = createFileReader('data.txt');
reader2.onDataReady(data => {
console.log('Second call data: ' + data);
});
});
After the output is like this:
First call data: some data
Let’s analyze why the second callback was not called:
In creatingreader1
At that time,inconsistentRead()
The function is executed asynchronously, and no cached results are available, so we have time to register the event listener. After the read operation is completed, it will be called in the next event cycle.
Then, it is created in the loop of the event loopreader2
, where the cache for the requested file already exists. In this case, the internal callinconsistentRead()
Will be synchronized. Therefore, its callback will be called immediately, which means thatreader2
All listeners of will also be called synchronously. However, in creatingreader2
After that, we started registering listeners, so they will never be called.
inconsistentRead()
The behavior of the callback function is unpredictable because it depends on many factors, such as the frequency of calls, the file name passed as a parameter, and the time it takes to load the file.
In practical applications, for example, the errors we just saw may be very complex and difficult to identify and copy in real applications. Imagine, inweb server
Using similar functions in, you can have multiple concurrent requests; Imagine that these requests are pending without any obvious reason and without any logs being recorded. This is definitely annoyingbug
.
npm
The founder of the and formerNode.js
Project leaderIsaac Z. Schlueter
In one of his blog articles, he compared the use of this unpredictable function to releaseZalgo
. If you are not familiar with itZalgo
. You can have a lookOriginal post by Isaac Z. Schlueter.
Use synchronization apis
From the above aboutzalgo
In the example of, we know that,API
Its nature must be clearly defined: synchronous or asynchronous?
We are suitable.fix
AboveinconsistentRead()
Function generatedbug
The way to do this is to make it block execution completely synchronously. And this is entirely possible becauseNode.js
For most basicI/O
The operation provides a set ofAPI
. For example, we can usefs.readFileSync()
Function to replace its asynchronous counterpart. The code is now as follows:
const fs = require('fs');
const cache = {};
function consistentReadSync(filename) {
if (cache[filename]) {
return cache[filename];
} else {
cache[filename] = fs.readFileSync(filename, 'utf8');
return cache[filename];
}
}
We can see that the entire function is transformed into a synchronous blocking call mode. If a function is synchronous, it will not beCPS
The style of. In fact, we can say that usingCPS
To achieve a synchronousAPI
It has always been a best practice, which will eliminate any confusion in its nature and will be more effective from a performance point of view.
Please remember, willAPI
FromCPS
Change to the style returned by direct call, or from asynchronous to synchronous. For example, in our example, we must completely change ourcreateFileReader()
To synchronize and adapt it to always work.
In addition, synchronization is usedAPI
Rather than asynchronousAPI
, pay special attention to the following matters needing attention:
- Synchronization
API
Not applicable to all application scenarios. - Synchronization
API
Loop blocking events and place concurrent requests in blocking state. It will destroyJavaScript
Even the performance of the entire application is degraded due to the concurrency model of. We will see the impact on our applications later in this book.
In ourinconsistentRead()
Function, because each file name is called only once, synchronous blocking calls will have little impact on the application, and cache values will be used for all subsequent calls. If the number of our static files is limited, then useconsistentReadSync()
It will not have a great impact on our event cycle. If we have a large number of files that need to be read once and have a high performance requirement, we do not recommend using theNode.js
Use synchronization inI/O
. However, in some cases, synchronizationI/O
May be the simplest and most effective solution. Therefore, we must correctly evaluate the specific application scenario to select the most appropriate scheme. The above example actually shows that synchronous blocking is used in actual applicationsAPI
Loading configuration files is very meaningful.
Therefore, remember to consider using synchronous blocking only if it does not affect the concurrency of the application.I/O
.
delayed time processing
Another kindfix
AboveinconsistentRead()
Function generatedbug
The way to do this is to make it only asynchronous. The solution here is to call synchronously in the next event cycle, instead of running immediately in the same event cycle, so that it is actually asynchronous. InNode.js
, you can use theprocess.nextTick()
Which delays the execution of the function until the next pass event loop. Its function is very simple, it takes a callback as a parameter and pushes it to the top of the event queue, in any unprocessedI/O
Before the event, and immediately return. Once the event loop runs again, the callback is called immediately.
So look at the following code, we can better use this technology to deal withinconsistentRead()
Asynchronous sequence of:
const fs = require('fs');
const cache = {};
function consistentReadAsync(filename, callback) {
if (cache[filename]) {
// 下一次事件循环立即调用
process.nextTick(() => callback(cache[filename]));
} else {
// 异步I/O操作
fs.readFile(filename, 'utf8', (err, data) => {
cache[filename] = data;
callback(data);
});
}
}
Now, the above function guarantees to call its callback function asynchronously under any circumstances, thus solving the above problem.bug
.
The other for delaying code executionAPI
YessetImmediate()
. Although their functions look very similar, their actual meanings are quite different.process.nextTick()
The callback function of theI/O
Called before the operation, whereas forsetImmediate()
Will be in otherI/O
Called after the operation. Due toprocess.nextTick()
In otherI/O
Previously called, so in some cases it may result inI/O
Enter an indefinite wait, such as a recursive callprocess.nextTick()
But forsetImmediate()
This will not happen. When we analyze using deferred calls to run synchronization later in this bookCPU
When binding tasks, we will learn more about the differences between the two apis.
We promise that through the use ofprocess.nextTick()
Call its callback function asynchronously.
Js callback style
ForNode.js
In general,CPS
StylishAPI
And callback functions follow a special set of conventions. These agreements are not only applicable toNode.js
CoreAPI
It is also meaningful for most user-level modules and applications that follow them. Therefore, we understand these styles and make sure we design asynchrony when needed.API
It is of utmost importance to abide by the regulations.
Callback is always the last argument
In all coresNode.js
In the method, the standard convention is that when a function accepts a callback in its input, it must be passed as the last argument. We use the followingNode.js
CoreAPI
For example:
fs.readFile(filename, [options], callback);
As can be seen from the previous example, even in the presence of optional parameters, callbacks are always placed at the last position. The reason is that in the case of callback definitions, function calls are more readable.
Error handling always comes first
InCPS
In, the error is passed in a callback function in a form different from the correct result. InNode.js
China,CPS
Any error generated by a style callback function is always passed as the first parameter of the callback, and any actual result is passed from the second parameter. If the operation is successful and there are no errors, the first parameter will benull
Orundefined
. Look at the following code:
fs.readFile('foo.txt', 'utf8', (err, data) => {
if (err)
handleError(err);
else
processData(data);
});
The above example is the best way to detect errors. If we do not detect errors, we may find it difficult to find and debug thebug
But another problem to consider is that mistakes are alwaysError
Type, which means that simple strings or numbers should not be passed as error objectstry catch
Code block capture).
Error propagation
As for the writing method of synchronous blocking, our mistakes are all throughthrow
Statement, even if the error jumps in the error stack, we can still capture the error context well.
But forCPS
Style of asynchronous calls, by passing the error to the next callback in the error stack to complete, the following is a typical example:
const fs = require('fs');
function readJSON(filename, callback) {
fs.readFile(filename, 'utf8', (err, data) => {
let parsed;
if (err)
// 如果有错误产生则退出当前调用
return callback(err);
try {
// 解析文件中的数据
parsed = JSON.parse(data);
} catch (err) {
// 捕获解析中的错误,如果有错误产生,则进行错误处理
return callback(err);
}
// 没有错误,调用回调
callback(null, parsed);
});
};
The details we notice from the above example are how to handle the exception correctly when we want tocallback
Pass parameters. In addition, when errors occur, we use thereturn
Statement, immediately exit the current function call, avoid the following related execution.
Unreachable exception
From the abovereadJSON()
Function to avoid throwing any exceptions to thefs.readFile()
In the callback function of theJSON.parse()
Place one aroundtry catch
Code block. Once an error occurs in an asynchronous callback, an exception will be thrown and the loop will jump to the event without propagating the error to the next callback function.
InNode.js
In, this is an unrecoverable state, and the application will close and print the error to standard output. In order to prove this point, we try to define from the previousreadJSON()
Delete from functiontry catch
Code block:
const fs = require('fs');
function readJSONThrows(filename, callback) {
fs.readFile(filename, 'utf8', (err, data) => {
if (err) {
return callback(err);
}
// 假设parse的执行没有错误
callback(null, JSON.parse(data));
});
};
In the above code, we have no way to captureJSON.parse
If we try to pass a nonstandardJSON
File format, will throw the following error:
SyntaxError: Unexpected token d
at Object.parse (native)
at [...]
at fs.js:266:14
at Object.oncomplete (fs.js:107:15)
Now, if we look at the previous error stack trace, we will see it fromfs
The module starts somewhere, just locally.API
Finished reading file and returned tofs.readFile()
Function, through the event loop. This information is clearly displayed to us, and the exception is passed from our callback to the stack, then directly into the event loop, and finally captured and thrown into the console.
This also means usingtry catch
Code block packing pairreadJSONThrows()
The call to will not work because the block is on a different stack than the call callback. The following code shows the opposite of what we just described:
try {
readJSONThrows('nonJSON.txt', function(err, result) {
// ...
});
} catch (err) {
console.log('This will not catch the JSON parsing exception');
}
previouscatch
Statements will never be receivedJSON
Parses the exception because it will return to the stack that threw the exception. We just saw the stack end in an event loop instead of triggering asynchronous operations.
As mentioned earlier, the application stopped at the moment when the exception reached the event loop, however, we still had the opportunity to perform some cleanup or logging before the application was terminated. In fact, when this happens,Node.js
A message nameduncaughtException
Special events of. The following code shows a sample use case:
process.on('uncaughtException', (err) => {
console.error('This will catch at last the ' +
'JSON parsing exception: ' + err.message);
// Terminates the application with 1 (error) as exit code:
// without the following line, the application would continue
process.exit(1);
});
What is important is that the uncaught exceptions will leave the application in a state where consistency cannot be guaranteed, which may lead to unforeseen problems. For example, it may be incompleteI/O
Requests to run or shut down may become inconsistent. This is why it is always recommended, especially in production environments, to write the above code for error logging after receiving an unhandled exception.
Module system and related modes
Modules are not only the basis for building large-scale applications, but their main mechanism is to encapsulate internal implementations, methods and variables through interfaces. In this section, we will introduceNode.js
The module system of and its most common usage mode.
About modules
JavaScript
One of the main problems of is that there is no namespace. Programs running in the global scope will pollute the global namespace and cause conflicts among related variables, data and method names. The technology to solve this problem is called module mode. Look at the following code:
const module = (() => {
const privateFoo = () => {
// ...
};
const privateBar = [];
const exported = {
publicFoo: () => {
// ...
},
publicBar: () => {
// ...
}
};
return exported;
})();
console.log(module);
This mode uses self-executing anonymous functions to implement the module and only exports the parts that are intended to be called publicly. In the above code, the module variable contains only the exportedAPI
And the rest of the module content is actually not accessible from outside. We will see later that the idea behind this model is used asNode.js
The foundation of the modular system.
Js module related explanation
CommonJS
It is an attempt to regulateJavaScript
The organization of the ecosystem, they put forwardCommonJS module specification
.Node.js
Based on this specification, its module system is built and some customized extensions are added. In order to describe its working principle, we can explain the module mode through such an example that each module runs under a private namespace, so that each variable defined in the module will not pollute the global namespace.
Custom module system
In order to explain the remoteness of the module system, let’s build a similar module system from scratch. The following code creates an impersonationNode.js
Originalrequire()
The function of a function.
Let’s first create a function that loads the contents of the module and wrap it in a private namespace:
function loadModule(filename, module, require) {
const wrappedSrc = `(function(module, exports, require) {
${fs.readFileSync(filename, 'utf8')}
})(module, module.exports, require);`;
eval(wrappedSrc);
}
The source code of the module is wrapped in a function, just like executing anonymous functions. The difference here is that we pass some inherent variables to the module, specificallymodule
,exports
Andrequire
. Note that the parameters of the export module aremodule.exports
Andexports
We will discuss it later.
Please remember, this is just an example, don’t do it in real projects. such aseval()
OrVm moduleIt may lead to some security problems, and others may exploit vulnerabilities to carry out injection attacks. We should use it very carefully or even avoid using it completely.eval
.
Let’s now look at how the module’s interfaces, variables, etc. arerequire()
Function to introduce:
const require = (moduleName) => {
console.log(`Require invoked for module: ${moduleName}`);
const id = require.resolve(moduleName);
// 是否命中缓存
if (require.cache[id]) {
return require.cache[id].exports;
}
// 定义module
const module = {
exports: {},
id: id
};
// 新模块引入,存入缓存
require.cache[id] = module;
// 加载模块
loadModule(id, module, require);
// 返回导出的变量
return module.exports;
};
require.cache = {};
require.resolve = (moduleName) => {
/* 通过模块名作为参数resolve一个完整的模块 */
};
The above function simulates the native for loading modulesNode.js
Therequire()
The behavior of a function. Of course, this is just onedemo
, it does not accurately and completely reflectrequire()
Function, but for better understandingNode.js
Internal implementation of the module system, defining modules and loading modules. The functions of our self-made module system are as follows:
- The module name is passed in as a parameter. The first thing we do is to find the complete path of the module. We call it
id
.require.resolve()
It is specially responsible for this function. It implements related functions through a specific parsing algorithm (to be discussed later). - If the module has already been loaded, it should exist in the cache. In this case, we immediately return to the modules in the cache.
- If the module has not been loaded, we will load the module for the first time. Creates a module object that contains a that is initialized with an empty object literal
exports
Property. This attribute will be used by the module’s code to derive the module’s commonAPI
. - Cache the first loaded module object.
- The module source code is read from its file, and the code is imported, as described earlier. We passed
require()
The function provides the module object we just created to the module. By operation or replacementmodule.exports
Object to export its public apis. - Finally, it will represent the commonality of the modules
API
Themodule.exports
Is returned to the caller.
As we can see,Node.js
The principle of the module system is not as profound as imagined, it is just that we create and import and export module source code through a series of operations.
Define a module
By viewing our customizationsrequire()
Function, now that we know how to define a module. Let’s look at the following example:
// 加载另一个模块
const dependency = require('./anotherModule');
// 模块内的私有函数
function log() {
console.log(`Well done ${dependency.username}`);
}
// 通过导出API实现共有方法
module.exports.run = () => {
log();
};
It should be noted that everything in the module is private unless it is assigned tomodule.exports
Variable. Then, when usingrequire()
When the module is loaded, the contents of this variable are cached and returned.
Define global variables
Even if all variables and functions declared in the module are defined within their local scope, global variables can still be defined. In fact, the module system discloses a system calledglobal
The special variable of. Everything assigned to this variable will be defined in the global environment.
Note: It is not good to pollute the global namespace, and the advantages of the module system are not fully utilized. Therefore, global variables should only be used if they really need to be used.
Exports module.exports
For many are not yet familiar withNode.js
For the developers of, they are most likely to confuse isexports
Andmodule.exports
To export publicAPI
The difference between. Variableexport
It’s just rightmodule.exports
Reference to the initial value of; As we have seen,exports
In essence, it is only a simple object before the module is loaded.
This means that we can only attach new attributes to objects referenced by exported variables, as shown in the following code:
exports.hello = () => {
console.log('Hello');
}
Give againexports
Assignment has no effect because it does not change.module.exports
The content of the, it just changed the variable itself. Therefore, the following code is wrong:
exports = () => {
console.log('Hello');
}
If we want to export something other than objects, such as functions, we can givemodule.exports
Reassign:
module.exports = () => {
console.log('Hello');
}
The require function is synchronized
Another important detail is what we wrote above.require()
Function is synchronous, it uses a relatively simple way to return the module contents, and does not need callback function. Therefore, formodule.exports
Also synchronized, for example, the following code is incorrect:
setTimeout(() => {
module.exports = function() {
// ...
};
}, 100);
Exporting modules in this way will have an important impact on how we define modules, because it limits the way we define and use modules synchronously. This is actually why the coreNode.js
The library provides synchronizationAPI
Instead of asynchronyAPI
One of the most important reasons.
If we need to define a module that requires asynchronous operation for initialization, we can also define and export the module that requires our asynchronous initialization at any time. However, we cannot guarantee that asynchronous modules are defined in this way.require()
After that, it can be used immediately. In Chapter 9, we will analyze this problem in detail and propose some models to optimize and solve this problem.
In fact, in the early daysNode.js
There was once an asynchronous version ofrequire()
However, due to its effect on initialization time and asynchronyI/O
The performance of the has a huge impact, soon thisAPI
It was deleted.
Resolve algorithm
Rely on hell
The dependence of software on software packages of different versions is described.Node.js
This problem is solved by loading different versions of modules, depending on the loading location of the modules. But all bynpm
To complete, the relevant algorithm is calledResolve algorithm
To be usedrequire()
In the function.
Now let’s give a quick overview of this algorithm. As described below,resolve()
The function returns a module name (moduleName
) as input and returns the full path of the module. The path is then used to load its code and can also uniquely identify the module.Resolve algorithm
There are three kinds of rules:
- File module: if
moduleName
In order to/
At the beginning, then it has been considered as the absolute path of the module. If by./
At the beginning, thenmoduleName
Considered a relative path, it is derived from the use ofrequire
The location of the module of the. - Core Module: If
moduleName
Not to/
Or./
At the beginning, the algorithm will first try to search in the core Node.js module. - Module package: if no match is found
moduleName
The core module of the, then search in the current directorynode_modules
If it is not foundnode_modules
, the search will continue to the upper directory.node_modules
Until it reaches the root directory of the file system.
For files and package modules, individual files and directories can also be matched tomoduleName
. In particular, the algorithm will attempt to match the following:
<moduleName>.js
<moduleName>/index.js
- In
<moduleName>/package.json
Themain
The file or directory declared under the value
Specific documents of resolve algorithm
node_modules
The catalog is actuallynpm
Place where each package is installed and related dependencies are stored. This means that based on the algorithm we just described, each package has its own private dependency. For example, look at the following directory structure:
myApp
├── foo.js
└── node_modules
├── depA
│ └── index.js
└── depB
│
├── bar.js
├── node_modules
├── depA
│ └── index.js
└── depC
├── foobar.js
└── node_modules
└── depA
└── index.js
In the previous example,myApp
,depB
AnddepC
All depend ondepA
; However, they all have their own version of private dependency! According to the rules of the analytic algorithm, use therequire('depA')
Different files will be loaded according to the required modules, as follows:
- In
/myApp/foo.js
Called in therequire('depA')
Will load/myApp/node_modules/depA/index.js
- In
/myApp/node_modules/depB/bar.js
Called in therequire('depA')
Will load/myApp/node_modules/depB/node_modules/depA/index.js
- In
/myApp/node_modules/depC/foobar.js
Called in therequire('depA')
Will load/myApp/node_modules/depC/node_modules/depA/index.js
Resolve algorithm
YesNode.js
The core part of dependency management, its existence makes even if the application has hundreds of packages, there will be no conflict and version incompatibility.
When we callrequire()
The analytic algorithm is transparent to us. However, you can still use therequire.resolve()
Directly used by any module.
Module cache
Each module will only be loaded when it is first introduced, and any time thereafterrequire()
Calls are taken from previously cached versions. By looking at our previously written customrequire()
Function, you can see that caching is crucial for performance improvement, and it also has some other advantages, as follows:
- Making it possible to reuse module dependencies
- To some extent, it is guaranteed that the same instance will always be returned when the same module is required from a given package, thus avoiding conflicts.
Module cache passrequire.cache
Variable view, so you can access it directly if you want. An example in practical application is through deletionrequire.cache
Variable to invalidate a cached module, which is very useful during testing, but can be very dangerous under normal circumstances.
Circular dependence
Many people think that circular dependence isNode.js
Internal design problems, but they can really happen in real projects, so we at least know how toNode.js
Make circular dependency effective. Let’s look at our customrequire()
Function, we can immediately see its working principle and matters needing attention.
Look at the following two modules:
- module
a.js
exports.loaded = false;
const b = require('./b');
module.exports = {
bWasLoaded: b.loaded,
loaded: true
};
- module
b.js
exports.loaded = false;
const a = require('./a');
module.exports = {
aWasLoaded: a.loaded,
loaded: true
};
Then we were inmain.js
Write the following code in:
const a = require('./a');
const b = require('./b');
console.log(a);
console.log(b);
Executing the above code prints the following results:
{
bWasLoaded: true,
loaded: true
}
{
aWasLoaded: false,
loaded: true
}
This result shows the processing sequence of cyclic dependency. Althougha.js
Andb.js
Both modules are fully initialized when the master module needs them, but when the slave moduleb.js
When loading,a.js
The module is incomplete. In particular, this state of affairs will continue untilb.js
The moment the load was completed. This kind of situation we should pay attention to, especially to confirm that we are inmain.js
The order required for the two modules in.
This is due to the modulea.js
You will receive an incomplete version of theb.js
. We now understand that if we lose control of which module to load first, and if the project is large enough, this may easily lead to circular dependency.
Documents on Circular References
In short, in order to prevent the dead cycle of module loading,Node.js
After the module is loaded for the first time, its results will be cached, and the results will be directly retrieved from the cache the next time it is loaded again. Therefore, in this circular dependency situation, there will be no dead loop, but the module will not be exported as expected due to caching (export
For detailed case analysis, see below).
The official website gives the simplest case that the three modules are not cyclic dependencies. In fact, the two modules can clearly express this situation. According to the idea of recursion, the simplest case is solved, and this kind of problem of arbitrary size is half solved (the other half needs to find out how the solution of the problem will change as the problem scale increases).
JavaScript
As an explanatory language, the above printout clearly shows the trajectory of the program. In this example,a.js
First of allrequire
Theb.js
Program entryb.js
Inb.js
In the first line againrequire
Thea.js
.
As mentioned earlier, in order to avoid infinite loop module dependence, inNode.js
runa.js
After that, it will be cached, but it should be noted that at this time the cache is only an unfinished one.a.js
(an unfinished copy of the a.js)。 So inb.js
Inrequire
Thea.js
When, the result is only an unfinished cachea.js
, specifically, it does not explicitly export the specific content (a.js
End). sob.js
Output froma
Is an empty object.
After that,b.js
After successful implementation, return toa.js
Therequire
Statement, continue execution to complete.
Module definition mode
The most common function of a module system is definition, besides its own mechanism for handling dependency relationships.API
. For definitionsAPI
, the main need to consider the balance between private and public functions. Its purpose is to maximize the internal realization and exposure of information hiding.API
Availability while balancing these with scalability and code reuse.
In this section, we will analyze some of the problems inNode.js
The most popular mode for defining modules in; Each module ensures transparency, extensibility and code reuse of private variables.
Named export
Expose the publicAPI
The most basic method of is to use named export, which includes assigning all the values we want to expose to theexport
(ormodule.exports
The property of the object referenced by the. In this way, the generated export object will become a container or namespace for a set of related functions.
Look at the following code, is the implementation of this pattern:
//file logger.js
exports.info = (message) => {
console.log('info: ' + message);
};
exports.verbose = (message) => {
console.log('verbose: ' + message);
};
The exported function is then used as an attribute of the module into which it is introduced, as shown in the following code:
// file main.js
const logger = require('./logger');
logger.info('This is an informational message');
logger.verbose('This is a verbose message');
MostNode.js
Modules use this definition.
CommonJS
The specification only allows useexports
Variable to the publicpublic
Members. Therefore, the named export mode is unique toCommonJS
Specification compatible mode. Usemodule.exports
YesNode.js
Provides an extension to support a broader module definition pattern.
Function derivation
One of the most popular module definition patterns includes integrating the entiremodule.exports
Variables are reassigned to a function. Its main advantage is that it only exposes one function, providing a clear entry point for the module, making it easier to understand and use, and it also well demonstrates the principle of single responsibility. This method of defining modules is also known in the community asSubstack mode
, view this mode in the following example:
// file logger.js
module.exports = (message) => {
console.log(`info: ${message}`);
};
This mode can also use the exported function as other publicAPI
The namespace of the. This is a very powerful combination because it still gives the module a separate entry point (exports
The main function of). This approach also allows us to expose other functions with secondary or higher-level use cases. The following code shows how to use the exported functions as namespaces to extend the modules we previously defined:
module.exports.verbose = (message) => {
console.log(`verbose: ${message}`);
};
This code demonstrates how to call the module we just defined:
// file main.js
const logger = require('./logger');
logger('This is an informational message');
logger.verbose('This is a verbose message');
Although it may be a limitation to export only one function, it is actually a perfect way to focus on a single function, which represents the most important function of this module, and at the same time makes the attributes of internal private variables more transparent, but only exposes the attributes of the exported function itself.
Node.js
The modularity of encourages us to follow the principle of single responsibility (SRP
): Each module should be responsible for a single function, which should be completely encapsulated by the module to ensure reusability.
Attention, what is said hereSubstack mode
, which is to expose the main functions of the module by exporting only one function. Use the exported function as a namespace to export other secondary functions.
Constructor (Class) Export
The module that derives the constructor is a special case of the module that derives the function. The difference is that with this new pattern, we allow users to create new instances using constructors, but we can also extend its prototype and create new classes (inheritance). The following is an example of this mode:
// file logger.js
function Logger(name) {
this.name = name;
}
Logger.prototype.log = function(message) {
console.log(`[${this.name}] ${message}`);
};
Logger.prototype.info = function(message) {
this.log(`info: ${message}`);
};
Logger.prototype.verbose = function(message) {
this.log(`verbose: ${message}`);
};
module.exports = Logger;
We use the above modules in the following ways:
// file main.js
const Logger = require('./logger');
const dbLogger = new Logger('DB');
dbLogger.info('This is an informational message');
const accessLogger = new Logger('ACCESS');
accessLogger.verbose('This is a verbose message');
viaES2015
Theclass
Keyword syntax can also implement the same pattern:
class Logger {
constructor(name) {
this.name = name;
}
log(message) {
console.log(`[${this.name}] ${message}`);
}
info(message) {
this.log(`info: ${message}`);
}
verbose(message) {
this.log(`verbose: ${message}`);
}
}
module.exports = Logger;
In view ofES2015
The class of is only the syntax sugar of the prototype, and the use of this module will be exactly the same as its prototype and constructor based scheme.
The export constructor or class is still the single entry point for the module, but is different from theSubstack mode
In comparison, it exposes more internal structure of the module. However, on the other hand, when we want to expand the function of this module, we can be more convenient.
Variants of this mode include those that do not usenew
The call to the. This little trick lets us use our module as a factory. Look at the following code:
function Logger(name) {
if (!(this instanceof Logger)) {
return new Logger(name);
}
this.name = name;
};
In fact, this is very simple: we checkthis
Does it exist and is itLogger
An example of. If any of these conditions arefalse
, it meansLogger()
Function is not usednew
, and then continue to correctly create the new instance and return it to the caller. This technology allows us to also use modules as factories:
// file logger.js
const Logger = require('./logger');
const dbLogger = Logger('DB');
accessLogger.verbose('This is a verbose message');
ES2015
Thenew.target
Grammar fromNode.js 6
At first, it provides a more concise method to realize the above functions. The utilization disclosednew.target
Property, which is available in all functionsMeta attribute
If you use thenew
Keyword calls a function, the calculation result at runtime istrue
.
We can rewrite the factory using this syntax:
function Logger(name) {
if (!new.target) {
return new LoggerConstructor(name);
}
this.name = name;
}
This code has exactly the same function as the previous code, so we can sayES2015
Thenew.target
Grammatical sugar makes the code more readable and natural.
Instance export
We can userequire()
To easily define stateful instances with states created from constructors or factories, which can be shared among different modules. The following code shows an example of this mode:
//file logger.js
function Logger(name) {
this.count = 0;
this.name = name;
}
Logger.prototype.log = function(message) {
this.count++;
console.log('[' + this.name + '] ' + message);
};
module.exports = new Logger('DEFAULT');
This newly defined module can be used as follows:
// file main.js
const logger = require('./logger');
logger.log('This is an informational message');
Because modules are cached, each needs toLogger
Modules of a module actually always retrieve the same instance of the object, sharing its state. This pattern is much like creating a singleton. However, it does not guarantee the uniqueness of the entire application instance, because it occurs in the traditional singleton pattern. When analyzing the parsing algorithm, we have actually seen that a module may be installed in the dependency tree of the application many times. This results in multiple instances of the same logical module, all running in the same instance.Node.js
In the context of the application. In chapter 7, we will analyze and export stateful instances and some alternative patterns.
The expansion of the model we just described includesexports
The constructor used to create the instance and the instance itself. This allows the user to create new instances of the same object or extend them if necessary. To achieve this, we only need to assign a new attribute to the instance, as shown in the following code:
module.exports.Logger = Logger;
We can then use the exported constructor to create other instances of the class:
const customLogger = new logger.Logger('CUSTOM');
customLogger.log('This is an informational message');
From the perspective of code availability, this is similar to using the exported function as a namespace. The module exports a default instance of an object, which is the function we use most of the time, while more advanced functions (such as the function of creating new instances or extending objects) can still be used with less exposed attributes.
Modify other modules or global scopes
A module can even export anything, which may seem a bit inappropriate; However, we should not forget that a module can modify the global scope and any objects therein, including other modules in the cache. Please note that these are usually considered bad practices, but because this mode may be useful and safe in some cases (e.g. testing), it is worth understanding and understanding that this feature can be used sometimes. We say that one module can modify other modules or objects in the global scope. It usually refers to temporary changes that modify existing objects at runtime to change or extend their behavior or applications.
The following example shows how we add a new function to another module:
// file patcher.js
// ./logger is another module
require('./logger').customMessage = () => console.log('This is a new functionality');
Write the following code:
// file main.js
require('./patcher');
const logger = require('./logger');
logger.customMessage();
In the above code, must first be introducedpatcher
Programs can only be usedlogger
Modules.
The above wording is very dangerous. The main consideration is that modules with modified global namespaces or other modules are operations with side effects. In other words, it will affect the state of entities outside its scope, which may lead to unpredictable consequences, especially when multiple modules interact with the same entity. Imagine that two different modules try to set the same global variable or modify the same attribute of the same module, and the effect may be unpredictable (which module wins? However, the most important thing is that it will affect the entire application.
observer mode
Node.js
Another important and basic mode in is the observer mode. AndReactor mode
, callback mode and module, observer mode isNode.js
One of the foundations is also the use of manyNode.js
The foundation of core modules and user-defined modules.
The observe mode is rightNode.js
The ideal solution for data response of is also a perfect complement to callback. We give the following definitions:
A publisher defines an object that can notify a group of observers (or listeners) when its state changes.
The main difference from callback mode is that the subject can actually notify multiple observers, whereas traditionalCPS
Style callbacks usually only propagate the main body’s results to a listener.
EventEmitter class
In traditional object-oriented programming, the observer pattern requires interfaces, specific classes, and hierarchical structures. InNode.js
In fact, it has become much simpler. Observer mode is already built into the core module and can be accessed throughEventEmitter
Class.EventEmitter
Class allows us to register one or more functions as listeners. When a specific event type is triggered, its callback will be called to notify its listeners. The following images intuitively explain this concept:
EventEmitter
Is a class (prototype) derived from the event core module. The following code shows how to get a reference to it:
const EventEmitter = require('events').EventEmitter;
const eeInstance = new EventEmitter();
EventEmitter
The basic method of is as follows:
-
on(event,listener)
: This method allows you to specify the type of eventString type
Register a new listener (a function) -
once(event, listener)
: This method registers a new listener and is then deleted after the event is first published -
emit(event, [arg1], [...])
: This method generates a new event and provides additional parameters to pass to the listener -
removeListener(event, listener)
: This method deletes the listener for the specified event type
All of the above methods will returnEventEmitter
Instance to allow linking. Listener functionfunction([arg1], [...])
So it only accepts the parameters provided when the event is issued. In the listener, this refers toEventEmitter
Generates an instance of the event.
We can see that a listener and a traditionalNode.js
Callbacks are very different; In particular, the first parameter is noterror
Which is passed to when calledemit()
Any data of.
Creating and using EventEmitter
Let’s look at how we use it in practice.EventEmitter
. The easiest way is to create a new instance and use it immediately. The following code shows how to use theEventEmitter
Realize the function of notifying subscribers in real time:
const EventEmitter = require('events').EventEmitter;
const fs = require('fs');
function findPattern(files, regex) {
const emitter = new EventEmitter();
files.forEach(function(file) {
fs.readFile(file, 'utf8', (err, content) => {
if (err)
return emitter.emit('error', err);
emitter.emit('fileread', file);
let match;
if (match = content.match(regex))
match.forEach(elem => emitter.emit('found', file, elem));
});
});
return emitter;
}
From the previous functionEventEmitter
Handle the three events that will occur:
-
fileread
Event: Triggered when a file is read -
found
Event: Triggered when the file content is successfully matched by regular matching -
error
Event: Triggered when there is an error reading the file
Look belowfindPattern()
How is the function triggered:
findPattern(['fileA.txt', 'fileB.json'], /hello \w+/g)
.on('fileread', file => console.log(file + ' was read'))
.on('found', (file, match) => console.log('Matched "' + match + '" in file ' + file))
.on('error', err => console.log('Error emitted: ' + err.message));
In the previous example, we areEventParttern()
FunctionEventEmitter
One listener is registered for each event type generated.
Error propagation
If the event is sent asynchronously,EventEmitter
You cannot throw an exception when an exception occurs, and the exception will be lost in the event loop. On the contrary, it isemit
Is to issue a special event called an error,Error object
Pass through parameters. This is exactly what we defined earlier.findPattern()
What is being done in the function.
For error events, it is always best practice to register listeners becauseNode.js
It will be handled in a special way, and if no associated listener is found, it will automatically throw an exception and exit the program.
Make any object visible
Sometimes, directly throughEventEmitter
Class to create a new observable object is not enough because nativeEventEmitter
Class does not provide us with the ability to expand the actual use of scenarios. We can expandEventEmitter
Class makes a common object observable.
To demonstrate this pattern, we try to implement it in an object.findPattern()
The function functions as shown in the following code:
const EventEmitter = require('events').EventEmitter;
const fs = require('fs');
class FindPattern extends EventEmitter {
constructor(regex) {
super();
this.regex = regex;
this.files = [];
}
addFile(file) {
this.files.push(file);
return this;
}
find() {
this.files.forEach(file => {
fs.readFile(file, 'utf8', (err, content) => {
if (err) {
return this.emit('error', err);
}
this.emit('fileread', file);
let match = null;
if (match = content.match(this.regex)) {
match.forEach(elem => this.emit('found', file, elem));
}
});
});
return this;
}
}
We definedFindPattern
The core module is used in the classutil
Providedinherits()
Function to expandEventEmitter
. In this way, it becomes an observable class that conforms to our actual application scenarios. The following is an example of its usage:
const findPatternObject = new FindPattern(/hello \w+/);
findPatternObject
.addFile('fileA.txt')
.addFile('fileB.json')
.find()
.on('found', (file, match) => console.log(`Matched "${match}"
in file ${file}`))
.on('error', err => console.log(`Error emitted ${err.message}`));
Now, through inheritanceEventEmitter
The function of, we can see nowFindPattern
In addition to being observable, the object has a complete set of methods.
This is inNode.js
In the ecosystem is a very common model, for example, the coreHTTP
modularServer
The object defines thelisten()
,close()
,setTimeout()
And internally it also inherits fromEventEmitter
Function to allow it to respond to requests related to events when it receives a new request, establishes a new connection, or the server shuts down.
extendEventEmitter
Other examples of objects for areNode.js
Flow. We will analyze it in more detail in chapter 5.Node.js
The flow of.
Synchronous and asynchronous events
Similar to callback mode, events also support synchronous or asynchronous sending. It is essential that we should never be in the same placeEventEmitter
The two methods are mixed in, but it is very important to consider synchronization or asynchrony when publishing the same event type, so as to avoid the inconsistency between synchronization and asynchronyzalgo
.
The main difference between publishing synchronous and asynchronous events is the way observers register. When an event is released asynchronously, even whenEventEmitter
After initialization, the program also registers a new observer, because it must ensure that this event is not triggered before the next cycle of the event cycle. As abovefindPattern()
Function. It represents the majorityNode.js
Common methods used in asynchronous modules.
On the contrary, synchronous publishing events require thatEventEmitter
The function must register the observer before starting to issue any event. Look at the following example:
const EventEmitter = require('events').EventEmitter;
class SyncEmit extends EventEmitter {
constructor() {
super();
this.emit('ready');
}
}
const syncEmit = new SyncEmit();
syncEmit.on('ready', () => console.log('Object is ready to be used'));
Ifready
If the event is issued asynchronously, the above code will run normally. However, since the event is issued synchronously and the listener is registered only after sending the event, the result does not call the listener, and the code will not print to the console.
Due to different application scenarios, it is sometimes used in a synchronous manner.EventEmitter
Functions are meaningful. Therefore, we must clearly highlight ourEventEmitter
To avoid unnecessary errors and exceptions.
Comparison of Event Mechanism and Callback Mechanism
When defining asynchronyAPI
When, the common difficulty is to check whether to useEventEmitter
Or accept only callback functions. The general distinction rule is this: when a result must be returned asynchronously, a callback function should be used, and when the result is needed and its way is uncertain, an event mechanism should be used to respond.
However, there is a lot of confusion because the two are so close and it is possible that both methods can achieve the same application scenario. Take the following code as an example:
function helloEvents() {
const eventEmitter = new EventEmitter();
setTimeout(() => eventEmitter.emit('hello', 'hello world'), 100);
return eventEmitter;
}
function helloCallback(callback) {
setTimeout(() => callback('hello world'), 100);
}
helloEvents()
AndhelloCallback()
It can be considered as equivalent in its function. The first uses event mechanism to implement, while the second uses callback to notify the caller and passes the event as a parameter. But what really distinguishes them is their executability, semantics and the amount of code to be implemented or used. Although we cannot give a set of definite rules to choose a style, of course we can provide some tips to help you make a decision.
Compared to the first example, observer mode, callback functions have some limitations in supporting different types of events. But in fact, we can still distinguish multiple events by passing the event type as a parameter of the callback or by accepting multiple callbacks. However, doing so cannot be considered an elegant one.API
. In this case,EventEmitter
It can provide better interface and simpler code.
EventEmitter
Another better application scenario is when the same event is triggered multiple times or no event is triggered. In fact, no matter whether the operation is successful or not, a callback is expected to be called only once. However, there is a special case where we may not know at which time the event is triggered. In this case,EventEmitter
Is the first choice.
Finally, use the callback’sAPI
Notify only specific callbacks, but useEventEmitter
Function allows multiple listeners to receive notifications.
The callback mechanism is used in combination with the event mechanism.
There are also cases where event mechanisms and callbacks can be combined. This pattern is particularly useful when we derive asynchronous functions.Node-glob moduleIs an example of this module.
glob(pattern, [options], callback)
The function takes a file name matching pattern as the first parameter, and the latter two parameters are a set of options and a callback function respectively. For the file list matching the specified file name matching pattern, the relevant callback function will be called. At the same time, the function returnsEventEmitter
Which shows the state of the current process. For example, it can be published in real time when the file name is successfully matched.match
Event, which can be published in real time when all the file lists are matched.end
Event, or when the process is manually abortedabort
Events. Look at the following code:
const glob = require('glob');
glob('data/*.txt', (error, files) => console.log(`All files found: ${JSON.stringify(files)}`))
.on('match', match => console.log(`Match found: ${match}`));
Summary
In this chapter, we first understand the difference between synchronous and asynchronous. Then, we discussed how to use callback mechanism and callback mechanism to deal with some basic asynchronous schemes. We have also learned the main differences between the two models and when they are more suitable for solving specific problems than the other model. We are only the first step towards a more advanced asynchronous mode.
In the next chapter, we will introduce more complex scenarios and learn how to use callback mechanism and event mechanism to deal with advanced asynchronous control problems.