Module Bundlers 101

Module Bundlers 101 | XCentium

> ## Excerpt > A Basic Primer for Understanding Module Bundlers

Why do we need them?

While using ES6 modules is natively supported in most browsers, it suffers from performance issues. Using a module bundler to handle your JavaScript modules is preferable in almost all cases. They also allow you to use modern JavaScript syntax without worrying about browser support. For example, you can write ES6+ code and have it transpiled down to ES5.

Note that module bundlers can also handle more than just JavaScript files. They have plugins that will allow you to import non-JavaScript assets such as CSS, images, and more.

Additionally, they can perform tree shaking - which in a nutshell means they can automatically strip out any unused code that would otherwise be included in your final bundle - to help keep your file size down.

Finally, they can be used for code splitting, which breaks down your code into multiple bundles. This way users are not forced to download ALL of your code at once, they only need to download the pieces they need for a given page or section.

How do they work?

For the sake of simplicity we’ll only dive into the core functionality of module bundlers, which is the process of bundling your JavaScript modules into production-ready code. Importing non-JS assets, tree shaking, and code splitting are outside the scope of this article.

The process of bundling code is typically done in 3 main steps:

Create an Abstract Syntax Tree
Generate a Dependency Graph
Build the Runtime

Step 1: Create an Abstract Syntax Tree

As the name implies, an abstract syntax tree (AST) is a tree-like structure that describes the syntax of source code. The easiest way to explain is to show an example.

If you head over to astexplorer.net, you can add some JavaScript source code in the left panel, which will use a JavaScript parser to generate an AST for you in the right panel. You can then click on different pieces of your source code - and it will highlight the corresponding node in the AST:

ast

… as you can see, every single piece of your code is described and/or accounted for in the AST. This is useful as it makes it easier to identify the import statements, which we'll need for the next step of the process where we generate a dependency graph:

![ast-import](https://www.xcentium.com/-/media/images/xccom-jpegs/blog-images/module-bundlers-101/ast-import.jpg?rev=836010b1820f43bb8021f2e9c30cf2a9&hash=8D08F353DE9C99ABE419B343E2F61ED4)

Step 2: Generate a Dependency Graph

A dependency graph is essentially an object (or group of objects) that maps each of your JavaScript modules to their dependencies. For example, let‘s say you have the following source code:

src/main.js

import { sayHello } from './message.js'; 

sayHello('Andrew');

src/message.js

function sayHello(name) {
    console.log(`Hello $&#123; name &#125;`);
}

export { sayHello };

Given the above, our dependency graph might look like the following:

[
    {
	id: 0,
        filename: './src/main.js',
        dependencies: ['./message.js'],
        mapping: {
            './message.js': 1,
        }
    },
    {
        id: 1,
        filename: './src/message.js',
        dependencies: [],
        mapping: {}
    }
]

Notice how each object in the array contains all of the information that we need - the module ID, the filename, and its dependencies.

This graph was generated by looping through the AST of each module, and looking for import statements to determine the relationship between modules.

Step 3: Build the Runtime

Finally, the module bundler will need to generate the runtime code that gets served to your clients. The exact implementation differs between libraries, but typically this can be done by:

Grabbing the source code of each module as a string, and concatenating them together.
Transpiling the code into CommonJS modules. This is needed since leaving the import and export statements as-is won’t work in the browser, and we can‘t override them. However by converting everything into CommonJS syntax - which uses require and module.exports instead - we can then inject custom functionality into those variables. In other words, by using const foo = require('foo') instead of import foo from 'foo', we can inject custom functionality for the former by writing require = somethingCustom().

To see this in action, you can head over to the Babel REPL. Enter your source code in the left panel, and you can view a transpiled version in the right panel.
Wrapping the source code in an IIFE, and injecting code to make the require and module.exports work they way we want (as mentioned in step 2).
Writing the output to a file.

Conclusion

Module bundlers have become commonplace for the typical front-end development workflow, and hopefully this post at least sheds a little bit of light on how the process works behind the scenes.

Granted, this is probably a lot to digest all at once, especially without seeing some code - so I’ve included a working example below for you to experiment with. Have fun!

src/main.js

import { sayHello } from './message.js'; sayHello('Andrew');

src/message.js

function sayHello(name) {    console.log(`Hello $&#123; name &#125;`);}export { sayHello };

src/bundler.js

// INSTRUCTIONS:
// 1) Add the sample code in the same directory as this file (e.g. main.js and message.js)
// 2) Run "npm install @babel/core @babel/preset-env @babel/traverse babylon -D"
// 3) Run "node bundler.js"


const fs = require('fs');
const path = require('path');
const babylon = require('babylon');
const traverse = require('@babel/traverse').default;
const babel = require('@babel/core');


const entryFile = 'main.js';
const outputFile = 'bundle.js';
let id = 0;


function createAsset(filename) {
    const content = fs.readFileSync(filename, 'utf-8');
    const ast = babylon.parse(content, { sourceType: 'module' });
    const dependencies = [];


    traverse(ast, {
        ImportDeclaration: ({ node }) => {
            dependencies.push(node.source.value);
        },
    });
    

    const { code } = babel.transformFromAst(ast, null, {
        presets: ['@babel/env'],
    });


    return {
        id: id++,
        filename,
        dependencies,
        code,
    };
}


function createGraph(entry) {
    const mainAsset = createAsset(entry);
    const queue = [mainAsset];


    for (const asset of queue) {
        const dirname = path.dirname(asset.filename);


        asset.mapping = {};


        asset.dependencies.forEach((relativePath) => {
            const absolutePath = path.join(dirname, relativePath);
            const child = createAsset(absolutePath);


            asset.mapping[relativePath] = child.id;
            queue.push(child);
        });
    }


    return queue;
}


function bundle(graph) {
    let modules = '';
    

    graph.forEach((mod) => {
        modules += `$&#123; mod.id &#125;: [
            function (require, module, exports) &#123;
                $&#123; mod.code &#125;
            &#125;,
            $&#123; JSON.stringify(mod.mapping) &#125;
        ],`;
    });


    const result = `
        (function(modules) &#123;
            function require(id) &#123;
                const [func, mapping] = modules[id];
                

                function localRequire(relativePath) &#123;
                    return require(mapping[relativePath]);
                &#125;
                

                const module = &#123; exports: &#123;&#125; &#125;;
                

                func(localRequire, module, module.exports);
                

                return module.exports;
            &#125;
            

            require(0);
        &#125;)(&#123; $&#123; modules &#125; &#125;)
    `;


    return result;
}


const graph = createGraph(entryFile);
const runtime = bundle(graph);


fs.writeFileSync(__dirname + `/$&#123; outputFile &#125;`, runtime);


console.log(`Success! Your bundle file (&quot;$&#123; outputFile &#125;&quot;) has been created.`);