There are many configuration tutorials for packaging tools, but do you know how they work?

  Front end, javascript, rollup, webpack

clipboard.png

Front-end modularity has become the mainstream today and cannot be separated from the contribution of various packaging tools. The introduction of webpack, rollup and the rising star parcel is endless in the community, and the analysis of their respective usage and configuration is also abundant. In order to avoid becoming a “configuration engineer”, we need to understand the operation principle of packaging tools. Only by understanding the core principle can we be more handy in the use of tools.

This article is based onparcelCore developers@ronamiThe Open Source Project ofminipackHowever, in order to make readers understand better, more understanding and explanation are added to the very detailed notes.

1. Core Principles of Packaging Tools

As the name implies, a packaging tool is a tool that is responsible for integrating some scattered small modules into a large module according to certain rules. At the same time, the packaging tool will also handle the dependencies between modules, and eventually this large module will be able to be run on the appropriate platform.

The packaging tool will start with an entry file, analyze the dependencies in it, and further analyze the dependencies in the dependencies, repeating the process until these dependencies are clarified and clarified.

From the above description, it can be seen that the most core part of the packaging tool is actually dealing with the dependencies between modules, and the minipack and the discussion in this article are also focused on the knowledge points of module dependencies.

For the sake of simplicity, the minipack project directly uses the ES modules specification. next we will create three new files and establish dependencies between them:

/* name.js */
 
 export const name = 'World'
/* message.js */
 
 import { name } from './name.js'
 
 export default `Hello ${name}!  `
/* entry.js */
 
 import message from './message.js'
 
 console.log(message)

Their dependency is very simple:entry.jsmessage.jsname.jsOf whichentry.jsIt will become the entry file of the packaging tool.

However, the dependency relationship here is only what we human beings understand. If we want machines to be able to understand the dependency relationship, we need to resort to certain means.

2. Dependency Analysis

Create a new js file namedminipack.js, first introduce the necessary tools.

/* minipack.js */
 
 const fs = require('fs')
 const path = require('path')
 const babylon = require('babylon')
 const traverse = require('babel-traverse').default
 const { transformFromAst } = require('babel-core')

Next, we will write a function that receives aFileAs a module, then read its contents and analyze all its dependencies. Of course, we can use theimportKeyword, but it is not very elegant, so we can use it.babylonThe js parser converts the contents of the file into an abstract syntax tree (AST) and directly obtains the information we need from the AST.

Once you have AST, you can use itbabel-traverseGo through the AST, get the key “dependency declaration” and save all the dependencies in an array.

Last usebabel-coreThetransformFromAstMethod collocationbabel-preset-envPlug-in, converts ES6 syntax into ES5 syntax recognizable by browser, and assigns an ID to js module.

let ID = 0
 
 function createAsset (filename) {
 //Read the contents of the file
 const content = fs.readFileSync(filename, 'utf-8')
 
 //Convert to AST
 const ast = babylon.parse(content, {
 sourceType: 'module',
 });
 
 //All dependencies of the file
 const dependencies = []
 
 //Get Dependency Statement
 traverse(ast, {
 ImportDeclaration: ({ node }) => {
 dependencies.push(node.source.value);
 }
 })
 
 //Convert ES6 syntax to ES5
 const {code} = transformFromAst(ast, null, {
 presets: ['env'],
 })
 
 //assign ID
 const id = ID++
 
 //Return to this module
 return {
 id,
 filename,
 dependencies,
 code,
 }
 }

runcreateAsset('./example/entry.js'), the output is as follows:

{ id: 0,
 filename: './example/entry.js',
 dependencies: [ './message.js' ],
 code: '"use strict";  \n\nvar _message = require("./message.js");  \n\nvar _message2 = _interopRequireDefault(_message);  \n\nfunction _interopRequireDefault(obj) { return obj && obj.__esModule ?   obj : { default: obj };   }\n\nconsole.log(_message2.default);'  }

Visibleentry.jsThe file has become a typical module and the dependencies have been analyzed. Next, we will recursively analyze the “dependencies in dependencies”, which is the atlas of establishing dependencies to be discussed in the next section.

3. Establish dependency atlas

Create a new one namedcreateGragh()Function, pass in the path of an entry file as a parameter, and then pass through thecreateAsset()Parse this file to define it as a module.

Next, in order to be able to do dependency analysis on modules one by one, we maintain an array, first pass the first module in and do analysis. When the module is analyzed to have other dependent modules, the dependent modules are also put into the array, and then the newly added modules are analyzed until all the dependencies and “dependencies in dependencies” are completely analyzed.

At the same time, we need to build a new modulemappingAttribute is used to store the dependency relationships among modules, dependencies and dependency IDs. for example, “module a with ID 0 depends on module b with ID 2 and module c with id 3” can be expressed as follows:

{
 0: [function A () {}, { 'B.js': 2, 'C.js': 3 }]
 }

Once you know the truth, you can start writing functions.

function createGragh (entry) {
 //Parse incoming files into modules
 const mainAsset = createAsset(entry)
 
 //Maintain an array and pass in the first module
 const queue = [mainAsset]
 
 //Traverse the array and analyze whether each module has other dependencies. If so, push the dependent modules into the array.
 for (const asset of queue) {
 asset.mapping = {}
 //Because the dependent paths are relative to the current module, all the relative paths should be treated as absolute paths.
 const dirname = path.dirname(asset.filename)
 //Traverse the dependencies of the current module and continue analysis
 asset.dependencies.forEach(relativePath => {
 //Construct absolute path
 const absolutePath = path.join(dirname, relativePath)
 //Generate Dependency Module
 const child = createAsset(absolutePath)
 //Write the dependency relation into the mapping of the module
 asset.mapping[relativePath] = child.id
 //Push this dependency module into the queue array to continue its analysis
 queue.push(child)
 })
 }
 
 //Finally return this queue, which is the dependency atlas.
 return queue
 }

There may be readers to one of themfor...of ...In circulationqueue.pushIt’s a bit confusing, but just try the following code to understand it:

var numArr = ['1', '2', '3']
 
 for (num of numArr) {
 console.log(num)
 if (num === '3') {
 arr.push('Done!'  )
 }
 }

Try to run itcreateGraph('./example/entry.js'), you can see the following output:

[ { id: 0,
 filename: './example/entry.js',
 dependencies: [ './message.js' ],
 code: '"use strict";  \n\nvar _message = require("./message.js");  \n\nvar _message2 = _interopRequireDefault(_message);  \n\nfunction _interopRequireDefault(obj) { return obj && obj.__esModule ?   obj : { default: obj };   }\n\nconsole.log(_message2.default);'  ,
 mapping: { './message.js': 1 } },
 { id: 1,
 filename: 'example/message.js',
 dependencies: [ './name.js' ],
 code: '"use strict";  \n\nObject.defineProperty(exports, "__esModule", {\n  value: true\n});  \n\nvar _name = require("./name.js");  \n\nexports.default = "Hello " + _name.name + "!"  ;  ',
 mapping: { './name.js': 2 } },
 { id: 2,
 filename: 'example/name.js',
 dependencies: [],
 code: '"use strict";  \n\nObject.defineProperty(exports, "__esModule", {\n  value: true\n});  \nvar name = exports.name = \'world\';'  ,
 mapping: {} } ]

Now that the dependency atlas has been built, the next step is to package them into a single file that can be run directly.

4, packaging

The dependency atlas generated in the previous step will be passed throughCommomJSSpecification to implement loading. Due to space constraints, this article is incorrect.CommomJSThe specification is expanded, and interested readers can refer to an article by @ Ruan YifengThe Principle and Implementation of Browser Loading CommonJS Module, said very clearly. Simply put, it is by constructing an immediate execution function(function () {})(), manually definedmodule,exportsAndrequireVariable, and finally achieve the purpose of code running in the browser.

The next step is to build code blocks by string splicing according to this specification.

function bundle (graph) {
 let modules = ''
 
 graph.forEach(mod => {
 modules += `${mod.id}: [
 function (require, module, exports) { ${mod.code} },
 ${JSON.stringify(mod.mapping)},
 ],`
 })
 
 const result = `
 (function(modules) {
 function require(id) {
 const [fn, mapping] = modules[id];
 
 function localRequire(name) {
 return require(mapping[name]);
 }
 
 const module = { exports : {} };
 
 fn(localRequire, module, module.exports);
 
 return module.exports;
 }
 
 require(0);
 })({${modules}})
 `
 return result
 }

Last runbundle(createGraph('./example/entry.js')), the output is as follows:

(function (modules) {
 function require(id) {
 const [fn, mapping] = modules[id];
 
 function localRequire(name) {
 return require(mapping[name]);
 }
 
 const module = { exports: {} };
 
 fn(localRequire, module, module.exports);
 
 return module.exports;
 }
 
 require(0);
 })({
 0: [
 function (require, module, exports) {
 "use strict";
 
 var _message = require("./message.js");
 
 var _message2 = _interopRequireDefault(_message);
 
 function _interopRequireDefault(obj) { return obj && obj.__esModule ?   obj : { default: obj };  }
 
 console.log(_message2.default);
 },
 { "./message.js": 1 },
 ], 1: [
 function (require, module, exports) {
 "use strict";
 
 Object.defineProperty(exports, "__esModule", {
 value: true
 });
 
 var _name = require("./name.js");
 
 exports.default = "Hello " + _name.name + "!"  ;
 },
 { "./name.js": 2 },
 ], 2: [
 function (require, module, exports) {
 "use strict";
 
 Object.defineProperty(exports, "__esModule", {
 value: true
 });
 var name = exports.name = 'world';
 },
 {},
 ],
 })

This code will be able to run directly in the browser and output “Hello world!” .

At this point, the whole packaging tool has been completed.

5. Summary

After the above steps, we can know a module packaging tool. The first step will start from the entry file and perform dependency analysis on it. The second step will recursively perform dependency analysis on all its dependencies again. The third step will build the dependency atlas of the module. The last step will be used according to the dependency atlas.CommonJSThe specification builds the final code. Knowing the purpose of each step in the process, one can understand the operation principle of a packaging tool.

Finally, thank you again@ronamiThe Open Source Project ofminipackThe source code has more detailed comments and is very worth reading.