“Node.js Design Pattern” Welcome to Node.js Platform

  javascript, node.js

This series of articles is《Node.js Design Patterns Second Edition》The translation of the original text and the reading notes were serialized and updated in GitHub.Synchronized translation links.

Welcome to pay attention to my column, and subsequent blog posts will be synchronized in the column:

Welcom to the Node.js Platform

The development of Node.js

  • The development of technology itself
  • hugeNode.jsDevelopment of Ecological Circle
  • Maintenance of Official Organizations

Features of Node.js

Small module

In order topackageIn principle, the capacity of each module should be as small and precise as possible.

Principles:

  • “Small is beautiful” — Small and Fine
  • “makeeach program do one thingwell”-principle of single responsibility

So, oneNode.jsThe application is built from multiple packages, and the package manager (npm) makes them depend on each other without conflict.

If you design aNode.jsThe module, as far as possible to do the following three points:

  • Easy to understand and use
  • Easy to test and maintain
  • Considering that support for clients (browsers) is more friendly

And,Don't Repeat Yourself(DRY)Principle of reusability.

Provided as an interface

Each ..Node.jsModules are all functions (classes are also presented in the form of constructors), and we only need to call correlationAPIThat is, there is no need to know the implementation of other modules.Node.jsModules are created to use them, not only in terms of extensibility, but also in terms of maintainability and usability.

Simple and practical

“Simplicity is Ultimate Complexity”-Darwin

FollowKiss principle, that is, excellent and concise design, can transfer information more effectively.

The design must be very simple, no matter in the implementation or the interface, what is more important is that the implementation is simpler than the interface, and simplicity is an important design principle.

We will make a software with simple design and complete functions instead of perfect ones:

  • It takes less effort to realize it.
  • Allows faster transportation of resources with less speed.
  • It is flexible and easier to maintain and understand.
  • Promote community contributions and allow the software itself to grow and improve.

And forNode.jsBecause of its supportJavaScript, simple and functions, closures, objects and other characteristics, can replace the complex object-oriented class syntax. Such as singleton pattern and decorator patterns, both require very complex implementations in object-oriented languages, while forJavaScriptIt is simpler.

Introduce the new syntax of Node.js 6 and ES2015

Let and const keywords

ES5Previously, there were only functions and global scopes.

if (false) {
  var x = "hello";
}

console.log(x); // undefined

Use nowletTo create a lexical scope, an error will be reportedUncaught ReferenceError: x is not defined

if (false) {
  let x = "hello";
}

console.log(x);

Use in Loop Statementslet, will also report an errorUncaught ReferenceError: i is not defined

for (let i = 0; i < 10; i++) {
  // do something here
}

console.log(i);

UseletAndconstKeyword, can make the code more secure, if accidentally access another scope of variables, it is easier to find errors.

UseconstKeyword declares variables that cannot be accidentally changed.

const x = 'This will never change';
x = '...';

A mistake will be reported here.Uncaught TypeError: Assignment to constant variable.

However, for changes to object properties,constAppear helpless:

const x = {};
x.name = 'John';

The above code will not report an error.

However, if the object is changed directly, an error will still be thrown.

const x = {};
x = null;

In actual use, we useconstIntroduce modules to prevent accidental changes:

const path = require('path');
let path = './some/path';

The above code will report an error, reminding us to change the module unexpectedly.

If you need to create immutable objects, it is only a simple use.constIs not enough, need to useObject.freeze()Ordeep-freeze

I looked at the source code, in fact very few, is recursive useObject.freeze()

module.exports = function deepFreeze (o) {
  Object.freeze(o);

  Object.getOwnPropertyNames(o).forEach(function (prop) {
    if (o.hasOwnProperty(prop)
    && o[prop] !== null
    && (typeof o[prop] === "object" || typeof o[prop] === "function")
    && !Object.isFrozen(o[prop])) {
      deepFreeze(o[prop]);
    }
  });
  
  return o;
};

Arrow function

Arrow functions are easier to understand, especially when we define callbacks:

const numbers = [2, 6, 7, 8, 1];
const even = numbers.filter(function(x) {
  return x % 2 === 0;
});

Use the arrow function syntax, more concise:

const numbers = [2, 6, 7, 8, 1];
const even = numbers.filter(x => x % 2 === 0);

If there is more than onereturnStatement uses the=> {}

const numbers = [2, 6, 7, 8, 1];
const even = numbers.filter((x) => {
  if (x % 2 === 0) {
    console.log(x + ' is even');
    return true;
  }
});

Most importantly, that arrow function bind its lexical scope, whichthisWith the parent code blockthisSame.

function DelayedGreeter(name) {
  this.name = name;
}

DelayedGreeter.prototype.greet = function() {
  setTimeout(function cb() {
    console.log('Hello' + this.name);
  }, 500);
}

const greeter = new DelayedGreeter('World');
greeter.greet(); // 'Hello'

To solve this problem, use the arrow function orbind

function DelayedGreeter(name) {
  this.name = name;
}

DelayedGreeter.prototype.greet = function() {
  setTimeout(function cb() {
    console.log('Hello' + this.name);
  }.bind(this), 500);
}

const greeter = new DelayedGreeter('World');
greeter.greet(); // 'HelloWorld'

Or arrow function, same scope as parent code block:

function DelayedGreeter(name) {
  this.name = name;
}

DelayedGreeter.prototype.greet = function() {
  setTimeout(() => console.log('Hello' + this.name), 500);
}

const greeter = new DelayedGreeter('World');
greeter.greet(); // 'HelloWorld'

Grammatical sugar

classIt is a grammatical sugar inherited from the prototype. For all developers from traditional object-oriented languages (such asJavaAndC#) is more familiar with, the new grammar has not changedJavaScriptThe running features of the are more convenient and easy to read through prototype.

Traditional passageConstructor+PrototypeThe wording of:

function Person(name, surname, age) {
  this.name = name;
  this.surname = surname;
  this.age = age;
}

Person.prototype.getFullName = function() {
  return this.name + '' + this.surname;
}

Person.older = function(person1, person2) {
  return (person1.age >= person2.age) ? person1 : person2;
}

UseclassGrammar is more concise, convenient and easy to understand:

class Person {
  constructor(name, surname, age) {
    this.name = name;
    this.surname = surname;
    this.age = age;
  }

  getFullName() {
    return this.name + '' + this.surname;
  }

  static older(person1, person2) {
    return (person1.age >= person2.age) ? person1 : person2;
  }
}

But the above implementations are interchangeable, however, forclassIn terms of grammar, the most significant thing isextendsAndsuperKey words.

class PersonWithMiddlename extends Person {
  constructor(name, middlename, surname, age) {
    super(name, surname, age);
    this.middlename = middlename;
  }

  getFullName() {
    return this.name + '' + this.middlename + '' + this.surname;
  }
}

This example is a true object-oriented approach. We declare a class that we want to inherit, define a new constructor, and use thesuperKeyword calls the parent constructor and overridesgetFullNameMethod to make it supportmiddlename.

New syntax for literal quantities of objects

Allowed defaults:

const x = 22;
const y = 17;
const obj = { x, y };

Allow method names to be omitted

module.exports = {
  square(x) {
    return x * x;
  },
  cube(x) {
    return x * x * x;
  },
};

Key’s calculation properties

const namespace = '-webkit-';
const style = {
  [namespace + 'box-sizing']: 'border-box',
  [namespace + 'box-shadow']: '10px 10px 5px #888',
};

New way to define getter and setter

const person = {
  name: 'George',
  surname: 'Boole',

  get fullname() {
    return this.name + ' ' + this.surname;
  },

  set fullname(fullname) {
    let parts = fullname.split(' ');
    this.name = parts[0];
    this.surname = parts[1];
  }
};

console.log(person.fullname); // "George Boole"
console.log(person.fullname = 'Alan Turing'); // "Alan Turing"
console.log(person.name); // "Alan"

Here, the second oneconsole.logTriggeredsetMethods.

Template string

Other ES2015 syntax

Reactor mode

Reactor modeYesNode.jsThe core module of asynchronous programming, its core concept is:Single threadNon-blocking I/OAs can be seen from the following examplesReactor modeInNode.jsThe embodiment of the platform.

I/O is slow

In the basic operation of a computer, I/O is definitely the slowest. The speed of accessing memory is nanosecond (10e-9 s), while accessing data on disk or accessing data on the network is slower, on the order of milliseconds (10e-3 s)。 The speed of memory transfer is generally considered to beGB/sTo calculate, however, disk or network access speed is relatively slow, generally isMB/s. Although forCPUIn general,I/OThe resource consumption of the operation is not too large, but it is being sentI/OThere is always a time delay between the request and the completion of the operation. In addition, we must also consider human factors. Usually, the input of the application program is artificially generated, such as the clicking of buttons and the sending of information by instant messaging tools. Therefore, the speed of input and output is not caused by the slow access rate of network and disk, and there are many factors.

Blocking I/O

In oneBlocking I/OIn the process of the model,I/OThe request blocks the subsequent code block. InI/OBefore the request operation is completed, the thread will have an indefinite period of time to waste. (It may be on the order of milliseconds, but it may even be on the order of minutes, such as when a user presses a key and does not release it). The following example is oneBlocking I/OModel.

// 直到请求完成,数据可用,线程都是阻塞的
data = socket.read();
// 请求完成,数据可用
print(data);

We know that,Blocking I/OThe server model of does not handle multiple connections in one thread, each timeI/OThe processing of other connections will be blocked. For this reason, for each concurrent connection that needs to be processed, the traditional web server’s processing method is to open a new process or thread (or reuse a process from the thread pool). In this way, when a thread due toI/OWhen an operation is blocked, it does not affect the availability of another thread because they are processed in separate threads.

Through the following picture:

From the above picture, we can see that each thread has been idle waiting for a period of time to receive new data from the associated connection. If all kinds ofI/OOperations block subsequent requests. For example, connecting databases and accessing file systems, now we can quickly know that a thread needs to waitI/OThe result of the operation waits for a lot of time. Unfortunately, a thread holdsCPUResources are not cheap; they consume memory and causeCPUContext switching, therefore, long-term possessionCPUHowever, threads that are not used most of the time are not efficient choices in terms of resource utilization.

Non-blocking I/O

Except ..Blocking I/OIn addition, most modern operating systems support another mechanism for accessing resources, namelyNon-blocking I/O. Under this mechanism, subsequent code blocks do not waitI/ORequest the return of data before execution. If all data is unavailable at the current time, the function returns a predefined constant value (for exampleundefined), indicating that no data is currently available.

For example, inUnixIn the operating system,fcntl()Function to operate an existing file descriptor and change its operation mode toNon-blocking I/O(AdoptedO_NONBLOCKStatus word). Once the resource is in non-blocking mode, if the read file operation has no readable data, or if the write file operation is blocked, the read operation or the write operation returns-1AndEAGAINWrong.

Non-blocking I/OThe most basic mode is to obtain data through polling, which is also calledBusy-wait model. Look at the following example, throughNon-blocking I/OAnd a polling mechanismI/OAs a result.

resources = [socketA, socketB, pipeA];
while(!resources.isEmpty()) {
  for (i = 0; i < resources.length; i++) {
    resource = resources[i];
    // 进行读操作
    let data = resource.read();
    if (data === NO_DATA_AVAILABLE) {
      // 此时还没有数据
      continue;
    }
    if (data === RESOURCE_CLOSED) {
      // 资源被释放,从队列中移除该链接
      resources.remove(i);
    } else {
      consumeData(data);
    }
  }
}

We can see that through this simple technology, different resources can already be processed in one thread, but it is still not efficient. In fact, in the previous example, the loop used to iterate resources would only consume valuableCPUAnd the waste of these resources compared withBlocking I/OOn the contrary, it is even more unacceptable. Polling algorithms usually waste a lot of money.CPUTime.

Event multiplexing

For obtaining non-blocking resources,Busy-wait modelNot an ideal technology. Fortunately, most modern operating systems provide a native mechanism to handle concurrency, and non-blocking resources (synchronous event multiplexers) are an effective method. This mechanism is called the event cycle mechanism. This event collection andI/O queueFromPublish-Subscribe Mode. The for the event multiplexer to collect resourcesI/OEvents and put them in the queue until they are processed. Look at the following pseudocode:

socketA, pipeB;
wachedList.add(socketA, FOR_READ);
wachedList.add(pipeB, FOR_READ);
while(events = demultiplexer.watch(wachedList)) {
  // 事件循环
  foreach(event in events) {
    // 这里并不会阻塞,并且总会有返回值(不管是不是确切的值)
    data = event.resource.read();
    if (data === RESOURCE_CLOSED) {
      // 资源已经被释放,从观察者队列移除
      demultiplexer.unwatch(event.resource);
    } else {
      // 成功拿到资源,放入缓冲池
      consumeData(data);
    }
  }
}

Three steps of event multiplexing:

  • Resources are added to a data structure to associate a specific operation for each resource, in this exampleread.
  • The event notifier consists of a group of observed resources. Once the event is about to trigger, synchronization will be calledwatchFunction and returns this event that can be processed.
  • Finally, each event returned by the event multiplexer is processed, at which time the event associated with the system resource will be read and non-blocking throughout the operation. Until all the events have been processed, the event multiplexer will block again, and then repeat this step, that isevent loop.

The above figure can well help us understand the use of synchronous time multiplexers and non-blocking in a single-threaded application.I/OAchieve concurrency. We can see that using only one thread does not affect our handling of multiple threads.I/OThe performance of the task. At the same time, we see that tasks are spread out over time in a single thread, rather than scattered among multiple threads. We see that tasks propagated in a single thread save the overall idle time of threads and are more beneficial for programmers to write code than tasks propagated in multiple threads. In this book, you can see that we can use a simpler concurrency strategy because there is no need to consider the mutual exclusion and synchronization of multithreading.

In the next chapter, we have more opportunities to discussNode.jsThe concurrency model of.

Introduce reactor mode

For nowReactor mode, it is designed by a special algorithm processing program (inNode.jsOnce the event is generated and processed in the event loop, it is relevanthandlerWill be called.

Its structure is shown in the figure:

Reactor modeThe steps of are:

  • The application generates a new by submitting the request to the time multiplexerI/OOperation. Application specificationhandler,handlerCalled after the operation is completed. Submitting a request to the event multiplexer is non-blocking, so its call will immediately return, returning the execution right to the application.
  • When a groupI/OWhen the operation is completed, the event multiplexer will add these new events to the event loop queue.
  • At this point, the event loop iterates over each event in the event loop queue.
  • For each event, the correspondinghandlerBe dealt with.
  • handlerIs part of the application code,handlerAfter the execution, the execution right will be returned to the event loop. However, inhandlerA new asynchronous operation may be requested during execution so that the new operation is added to the event multiplexer.
  • When all the events in the event loop queue have been processed, the loop will block again in the event multiplexer until a new event can be processed to trigger the next loop.

We can now defineNode.jsThe core model of:

Mode (Reactor) Blocking TreatmentI/OUntil there are new events to be processed in a group of observed resources, then each event is assigned to correspond to.handlerThe way we react.

OS non-blocking I/O engine

Each operating system has its own interface to the event multiplexer,LinuxYesepoll,Mac OSXYeskqueue,WindowsTheIOCP API. Except that even in the same operating system, eachI/OOperations behave differently for different resources. For example, inUnixHowever, normal file systems do not support non-blocking operations, so in order to simulate non-blocking behavior, it is necessary to use an independent thread outside the event loop. All these intra-platform and cross-platform inconsistencies need to be abstracted at the upper level of the event multiplexer. That’s why.Node.jsIn order to be compatible with all mainstream platforms
Writing c language librarylibuv, the purpose is to makeNode.jsCompatible with all mainstream platforms and standardize non-blocking behavior of different types of resources.libuvToday asNode.jsTheI/OThe bottom of the engine.