Application containers have emerged as a powerful tool in modern software development. Lighter and more resource efficient than traditional virtual machines, containers offer IT organizations new opportunities in version control, deployment, scaling, and security.

This post will address what exactly containers are, why they are proving to be so advantageous, how people are using them, and best practices for containerizing your Node.js applications with Docker.

What’s a Container?

Put simply, containers are running instances of container images. Images are layered alternatives to virtual machine disks that allow applications to be abstracted from the environment in which they are actually being run. Container images are executable, isolated software with access to the host's resources, network, and filesystem. These images are created with their own system tools, libraries, code, runtime, and associated dependencies hardcoded. This allows for containers to be spun up irrespective of the surrounding environment. This everything-it-needs approach helps silo application concerns, providing improved systems security and a tighter scope for debugging.

Unlike traditional virtual machines, container images give each of its instances shared access to the host operating system through a container runtime. This shared access to the host OS resources enables performance and resource efficiencies not found in other virtualization methods.

Imagine a container image that requires 500 mb. In a containerized environment, this 500 mb can be shared between hundreds of containers assuming they are are all running the same base image. VMs, on the other hand, would need that 500 mb per virtual machine. This makes containers much more suitable for horizontal scaling and resource-restricted environments.

Why Application Containers?

The lightweight and reproducible nature of containers have made them an increasingly favored option for organizations looking to develop software applications that are scalable, highly available, and version controlled.

Containers offer several key advantages to developers:

• Lightweight and Resource Efficient. Compared to VMs, which generate copies of their host operating system for each application or process, containers have significantly less of an impact on memory, CPU usage, and disk space.
• Immutable. Containers are generated from a single source of truth, an image. If changes are committed to an image, a new image is made. This makes container image changes easy to track, and deployment rollbacks intuitive. The reproducibility and stability of containers helps development teams avoid configuration drift, making things like version testing and mirroring development and production environments much simpler.
• Portable. The isolated and self-reliant nature of containers makes them a great fit for applications that need to operate across a host of services, platforms, and environments. They can run on Linux, Windows, and macOS. Provide them from the cloud, on premise, or wherever your infrastructure dictates.
• Scalable and Highly Available. Containers are easily reproducible and can be made to dynamically respond to traffic demands, with orchestration services such as Azure Container Instances, Google Cloud Engine, and Amazon ECS making it simpler than ever to generate or remove containers from your infrastructure.

Application Container Use Cases

Not all applications and organizations are going to have the same infrastructure requirements. The aforementioned benefits of containers make them particularly adept at addressing the following needs:

DevOps Organizations

For teams working to practice ‘infrastructure as code’ and seeking to embrace the DevOps paradigm, containers offer unparalleled opportunities. Their portability, resistance to configuration drift, and quick boot time make containers an excellent tool for quickly and reproducibly testing different code environments, regardless of machine or location.

Microservice and Distributed Architectures

A common phrase in microservice development is “do one thing and do it well,” and this aligns tightly with application containers. Containers offer a great way to wrap microservices and isolate them from the wider application environment. This is very useful when wanting to update specific (micro-)services of an application suite without updating the whole application.

A/B testing

Containers make it easy to roll out multiple versions of the same application. When coupled with incremental rollouts, containers can keep your application in a dynamic, responsive state to testing. Want to test a new performance feature? Spin up a new container, add some updates, route 1% of traffic to it, and collect user and performance feedback. As the changes stabilize and your team decides to apply it to the application at large, containers can make this transition smooth and efficient.

Containers and Node.js

Because of application containers suitability for focused application environments, Node.js is arguably the best runtime for containerization.

• Explicit Dependencies. Containerized Node.js applications can lock down dependency trees, and maintain stable package.json, package-lock.json, or npm-shrinkwrap.json files.
• Fast Boot and Restart. Containers are lightweight and boot quickly, making them a strategic pair for Node.js applications. One of the most lauded features of Node.js is its impressive startup time. This robust boot performance gets terminated processes restarted quickly and applications stabilized; containerization provides a scalable solution to maintaining this performance.
• Scaling at the Process Level. Similar to the Node.js best practice of spinning up more processes instead of more threads, a containerized environment will scale up the number of processes by increasing the number of containers. This horizontal scaling creates redundancy and helps keep applications highly available, without the significant resource cost of a new VM per process.

Dockerizing Your Node.js Application

Docker Overview

Docker is a layered filesystem for shipping images, and allows organizations to abstract their applications away from their infrastructure.

With Docker, images are generated via a Dockerfile. This file provides configurations and commands for programmatically generating images.

Each Docker command in a Dockerfile adds a ‘layer’. The more layers, the larger the resulting container.

Here is a simple Dockerfile example:

1    FROM node:8
2 
3    WORKDIR /home/nodejs/app
4
5    COPY . .
6    RUN npm install --production
7
8    CMD [“node”, “index.js”]

The FROM command designates the base image that will be used; in this case, it is the image for Node.js 8 LTS release line.

The RUN command takes bash commands as its arguments. In Line 2 we are creating a directory to place the Node.js application. Line 3 lets Docker know that the working directory for every command after line 3 is going to be the application directory. Line 5 copies everything the current directory into the current directory of the image, which is /home/nodejs/app previously set by the WORKDIR command in like 3. On Line 6, we are setting up the production install.

Finally, on line 8, we pass Docker a command and argument to run the Node.js app inside the container.

The above example provides a basic, but ultimately problematic, Dockerfile.

In the next section we will look at some Dockerfile best practices for running Node.js in production.

Dockerfile Best Practices

Don’t Run the Application as root

Make sure the application running inside the Docker container is not being run as root.

1   FROM node:8
2
3   RUN groupadd -r nodejs && useradd -m -r -g -s /bin/bash nodejs nodejs
4
5   USER nodejs
6
7   ...

In the above example, a few lines of code have been added to the original Dockerfile example to pull down the image of the latest LTS version of Node.js, as well as add and set a new user, nodejs. This way, in the event that a vulnerability in the application is exploited, and someone manages to get into the container at the system level, at best they are user nodejs which does not have root permissions, and does not exist on the host.

Cache node_modules

Docker builds each line of a Dockerfile individually. This forms the 'layers' of the Docker image. As an image is built, Docker caches each layer.

7   ...
8     WORKDIR /home/nodejs/app
9
10  COPY package.json .
12  RUN npm install --production
13  COPY . . 
14
15  CMD [“node.js”, “index.js”]
16  ...

On line 10 of the above Dockerfile, the package.json file is being copied to the working directory established on line 8. After the npm install on line 12, line 13 copies the entire current directory into the working directory (the image).

If no changes are made to your package.json, Docker won’t rebuild the npm install image layer, which can dramatically improve build times.

Setup Your Environment

It’s important to explicitly set any environmental variables that your Node.js application will be expecting to remain constant throughout the container lifecycle.

12  ...
13  COPY . .
14  
15  ENV  NODE_ENV production
16
17  CMD [“node.js”, “index.js”]
18

With aims of comprehensive image and container services, DockerHub “provides a centralized resource for container image discovery, distribution and change management, user and team collaboration, and workflow automation throughout the development pipeline.”

To link the Docker CLI to your DockerHub account, use docker login:

docker login [OPTIONS] [SERVER]

Private GitHub Accounts and npm Modules

Docker runs its builds inside of a sandbox, and this sandbox environment doesn’t have access to information like ssh keys or npm credentials. To bypass this constraint, there are a couple recommended options available to developers:

• Store keys and credentials on the CI/CD system. The security concerns of having sensitive credentials inside of the Docker build can be avoided entirely by never putting them in there in the first place. Instead, store them on and retrieve them from your infrastructure’s CI/CD system, and manually copy private dependencies into the image.
• Use an internal npm server. Using a tool like Verdaccio, setup an npm proxy that keeps the flow of internal modules and credentials private.

Be Explicit with Tags

Tags help differentiate between different versions of images. Tags can be used to identify builds, teams that are working on the image, and literally any other designation that is useful to an organization for managing development of and around images. If no tag is explicitly added, Docker will assign a default tag of latest after running docker build. As a tag, latestis okay in development, but can be very problematic in staging and production environments.

To avoid the problems around latest, be explicit with your build tags. Here is an example script assigning tags with environment variables for the build’s git sha, branch name, and build number, all three of which can be very useful in versioning, debugging, and deployment management:

1   # !/bin/sh
2   docker tag helloworld:latest yourorg/helloworld:$SHA1
3   docker tag helloworld:latest yourorg/helloworld:$BRANCH_NAME
4   docker tag helloworld:latest yourorg/build_$BUILD_NUM
5

Read more on tagging here.

Containers and Process Management

Containers are designed to be lightweight and map well at the process level, which helps keep process management simple: if the process exits, the container exits. However, this 1:1 mapping is an idealization that is not always maintained in practice.

As Docker containers do not come with a process manager included, add a tool for simple process management.

dumb-init from Yelp is a simple, lightweight process supervisor and init system designed to run as PID 1 inside container environments. This PID 1 designation to the dumb-init process is normally assigned to a running Linux container, and has its own kernel-signaling idiosyncrasies that complicate process management. dumb-init provides a level of abstraction that allows it to act as a signal proxy, ensuring expected process behavior.

What to Include in Your Application Containers

A principal advantage of containers is that they provide only what is needed. Keep this in mind when adding layers to your images.

Here is a checklist for what to include when building container images:

• Your application code and its dependencies.
• Necessary environment variables.
• A simple signal proxy for process management, like dumb-init.

That’s it.

Conclusion

Containers are a modern virtualization solution best-suited for infrastructures that call for efficient resource sharing, fast startup times, and rapid scaling.

Application containers are being used by DevOps organizations working to implement “infrastructure as code,” teams developing microservices and relying on distributed architectures, and QA groups leveraging strategies like A/B testing and incremental rollouts in production.

Just as the recommended approach for single-threaded Node.js is 1 process: 1 application, best practice for application containers is 1 process: 1 container. This mirrored relationship arguably makes Node.js the most suitable runtime for container development.

Docker is an open platform for developing, shipping, and running containerized applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. When using Docker with Node.js, keep in mind:

• Don’t run the application as root
• Cache node_modules
• Use your CI/CD system or an internal server to keep sensitive credentials out of the container image
• Be explicit with build tags
• Keep containers light!

One Last Thing

If you’re interested in deploying Node.js applications within Docker containers, you may be interested in N|Solid. We work to make sure Docker is a first-class citizen for enterprise users of Node.js who need insight and assurance for their Node.js deployments.

Deploying N|Solid with Docker is as simple as changing your FROM statement! If you’d like to tune into the world of Node.js, Docker, Kubernetes, and large-scale Node.js deployments, be sure to follow us at @NodeSource on Twitter.

As part of the NodeSource Support team, I spend much of my time helping our customers analyze and resolve complex issues in Node.js. While factors like architecture and environment mean that some issues are quite unique, there are some familiar struggles that we’ve seen repeatedly from a wide variety of customers. I’ve listed a few of these common and relatively easy-to-avoid challenges below, along with our recommended strategy for avoiding (or resolving) these issues, as I think this information could help more teams working with Node.js avoid major headaches.

Sharing components, modules, or libraries between projects

Issue

The Support team frequently sees questions about the most effective way to share the components, models, and/or libraries between projects. In some cases, our customers are already using Flow and Stampit, which are useful tools for ReactJs components, and they’re looking for tools with less complexity built for Node.js codebases.

Answer

When this question comes up, we usually recommend turning each component (or model, or library) into a module and listing these in each project’s package.json file. This allows teams to share code across unified codebases by re-using localized modules.

Importing these components to a project can be accomplished with a fairly simple addition to the project’s package.json file:

“db-models”: “file:../mainproject/models”,

To use this approach, make sure you are using npm@5, or use the linklocal package for earlier npm versions.

Fixing cookie violations in Node.js with Express

Issue

Many teams have web scans to identify and analyze cookie violations in their Node.js environments when Express is also part of their tech stack. Some of the most common cookie violations found are:

• Cookie No httpOnly Flag
• Cookie Without secure Flag: if set to true, “the browser will not send a cookie with the secure flag set over an unencrypted HTTP request”

We’re frequently asked how best to set the HttpOnly and secure flags for cookies, and whether that can be done at the server level.

Answer

The default cookie settings in Express aren’t highly secure; however, these settings can be manually tightened to enhance security - for both an application and its users.

• httpOnly - This should be set to “true”-- Flags cookies to be accessible by the issuing web server, which assists in preventing session hijacking.
• secure - This should be set to “true”-- which requires TLS/SSL -- to allow the cookie to only be used with HTTPS requests, and not insecure HTTP requests. For more information about how to deal with this issue, I recommend checking out these two blog posts:
• 9 Security Tips to Keep Express from Getting Pwned - NodeSource
• Security Best Practices for Express in Production

Migrate a Java service to Node.js for specific tasks

Issue

We often talk to teams who are working to migrate individual tasks or functionality from Java into a microservices-oriented Node.js application. The best practices approach is to replace a single, monolithic Java app with multiple Node.js apps, each of which is dedicated to a specific task or closely-related set of tasks. We’re frequently asked to recommend a library or pattern that will allow a Node.js app to read from an OracleDB and push to an MQ-enabled application.

Answer

To connect to an OracleDB, we recommend the node-oracledb package, which is developed and maintained by Oracle and includes detailed documentation and examples.

There are a number of ways to access MQ from Node.js, depending on your needs:

• Take a look at the MQ Light client. MQ supports connections from MQ Light clients via AMQP channels.
• MQTT.js is a client library for the MQTT protocol. MQ supports connections from MQTT clients via the XR service and Telemetry channels.
• MQ versions 9.0.4 and higher include a simple REST API for messaging that is accessible from any environment.

When migrating from a Java project or starting a new Node.js project we also recommend: Don’t use console.log or console.error; instead utilize an abstraction library like Winston to control logging levels. Set the ability to adjust logging level using env variables

Identify the most appropriate package for your needs.

Issue

The npm registry contains more than 800,000 packages, so it’s not surprising that teams have a hard time deciding which package offers both the features and functionality as well as the level of security that is most suitable for their Node.js applications. Among the most common asks we see are recommendations related to creating pdfs, managing RxJS and Promises, and setting up proxy servers and http error handling. That said, needs vary wildly by project, so the advice below is intended to be generally applicable.

Answer

There are a few tools in the Node ecosystem that allow easy checking for vulnerabilities in Node.js application dependencies. These tools are highly valuable in as they can ensure that the packages installed in an application have no known vulnerabilities, and can prevent the installation of package updates if a vulnerability has been detected in a more recent package version.

• Snyk: Website - GitHub
• Node Security Project: Website - GitHub
• Retire.js: Website - GitHub

Once basic security checks have been passed, we recommend looking for the following factors to help you decide which package is best:

• Consistent, ongoing activity in the source code repository
• Acceptable resolution time for issues reported by other users (what is “acceptable” might vary depending on your application and needs)
• The frequency (and recency) of releases which include fixes and new features are released
• Presence of up-to-date documentation of the package code and features, ideally with examples
• Do a general code check to ensure all the features you’ll need are working as expected

Increase the data submission limit in HTTP responses

Issue

For teams using Node.js and Express, we often hear that a POST request containing a large body of JSON is returning a 413: Payload Too Large response. Most of the time, the engineers we talk to want to know how to safely increase the size limit of the request body.

Answer

There are multiple ways to safely increase the size limit of the request body.

For a quick fix, either of the following two options would work:

app.use(BodyParser.json({ limit: ‘50mb’, type: ‘application/json’ }))

app.use(BodyParser.urlencoded({ limit: ‘50mb’, ‘extended’: ‘true’, parameterLimit: 50000 }));

Both of the above examples raise the maximum size of the request body to 50mb; in the second example, the parameterLimit value is also defined.

While a quick fix will work, implementing a caching solution with Redis is a good option too. The idea is to store the data in cache and then send a reference from the client to the data instead of sending a big payload.

Similarly, you will not need to receive back a massive amount of data in JSON format; instead, you send the reference from the client and retrieve the cached info at the backend. This allows comparatively lightweight requests and avoids a negative impact on the performance of the application.

Hopefully the suggestions above help your team resolve (or avoid entirely) some of the most common issues reported by our Node.js Support customers. If you have questions or are experiencing an issue not covered above, feel free to contact us on Twitter @NodeSource, or consider joining one of our upcoming Office Hours sessions, which are hosted by one of our senior Solutions Architects and open to anyone with Node.js-related questions.

This article was first published in NodeSource blog on March 2017.

The package.json file is core to the Node.js ecosystem and is a basic part of understanding and working with Node.js, npm, and even modern JavaScript. The package.json is used as what equates to a manifest about applications, modules, packages, and more - it's a tool to that's used to make modern development streamlined, modular, and efficient.

As a developer in the Node.js ecosystem, understanding the basics of package.json is one of the first steps to really kicking off your development experience with Node.js.

Because of how essential understanding the basics of package.json is to development with Node.js, I've gone through and outlined some of the most common and important properties of a package.json file that you'll need to use package.json effectively.

Identifying Metadata Inside package.json

The name property

The name property in a package.json file is one of the fundamental components of the package.json structure. At its core, name is a string that is exactly what you would expect - the name of the module that the package.json is describing.

Inside your package.json, the name property as a string would look something like this:

    "name": "metaverse"

Despite having only a few material restrictions (a max length of 214 characters, can't begin with . or _, no uppercase letters, and no characters that aren't URL-friendly), one interesting aspect of the name property is that, there have been software ecosystems that have developed standard naming conventions that enable discoverability simply by using the name property.

A few examples of this kind of namespacing are babel-plugin- for Babel and the the webpack -loader tooling.

The version property

The version property is a key part of a package.json, as it denotes the current version of the module that the package.json file is describing.

While the version property isn't required to follow semver, semver is the standard used by the vast majority of modules and projects in the Node.js ecosystem - and the module version, according to semver, is what you'll typically find in the version property of a package.json file.

Inside your package.json, the version property as a string using semver could look like this:

    "version": "5.12.4"

The license property

The license property of a package.json file is used to note what license the module that the package.json file is describing. While there are some complex ways you can use the license property of a package.json file (to do things like dual-licensing or defining your own license), the most typical usage of it is to use a SPDX License identifier - some examples that you may recognize are MIT, ISC, and GPL-3.0.

Inside your package.json, the license property with an MIT license look like this:

    "license": "MIT"

The description property

The description property of a package.json file is a string that contains a human-readable description about the module - basically, it's the module developer's chance to quickly let users know what exactly a module does. The description property is frequently indexed by search tools like npm search and the npm CLI search tool to help find relevant packages based on a search query.

Inside your package.json, the description property would look like this:

    "description": "The Metaverse virtual reality. The final outcome of all virtual worlds, augmented reality, and the Internet."

The keywords property

The keywords property inside a package.json file is, as you may have guessed, a collection of keywords about a module. Keywords can help identify a package, related modules and software, and concepts.

The keywords property is always going to be an array, with one or more strings as the array's values - each one of these strings will, in turn, be one of the project's keywords.

Inside your package.json, the keywords array would look something like this:

    "keywords": [
        "metaverse",
        "virtual reality",
        "augmented reality",
        "snow crash"
    ]

Functional Metadata Inside package.json

The main property

The main property of a package.json is a direction to the entry point to the module that the package.json is describing. In a Node.js application, when the module is called via a require statement, the module's exports from the file named in the main property will be what's returned to the Node.js application.

Inside your package.json, the main property, with an entry point of app.js, would look like this:

    "main": "app.js",

The repository property

The repository property of a package.json is an array that defines where the source code for the module lives. Typically, for open source projects, this would be a public GitHub repo, with the repository array noting that the type of version control is git, and the URL of the repo itself. One thing to note about this is that it's not just a URL the repo can be accessed from, but the full URL that the version control can be accessed from.

Inside your package.json, the repository property would look like this:

  "repository": {
      "type": "git",
      "url": "https://github.com/bnb/metaverse.git"
  }

The scripts property

The scripts property of a package.json file is simple conceptually, but is complex functionally to the point that it's used as a build tool by many.

At its simplest, the scripts property takes an object with as many key/value pairs as desired. Each one of the keys in these key/value pairs is the name of a command that can be run. The corresponding value of each key is the actual command that is run. Scripts are frequently used for testing, building, and streamlining of the needed commands to work with a module.

Inside your package.json, the scripts property with a build command to execute node app.js (presumably to build your application) and a test command using Standard would look like this:

    "scripts": {
        "build": "node app.js",
        "test": "standard"
    }

The dependencies property

The dependencies property of a module's package.json is where dependencies - the other modules that this module uses - are defined. The dependencies property takes an object that has the name and version at which each dependency should be used. Tying things back to the version property defined earlier, the version that a module needs is defined. Do note that you'll frequently find carets (^) and tildes (~) included with package versions. These are the notation for version range - taking a deep-dive into these is outside the scope of this article, but you can learn more in our primer on semver.

Inside your package.json, the dependencies property of your module may look something like this:

  "dependencies": {
    "async": "^0.2.10",
    "npm2es": "~0.4.2",
    "optimist": "~0.6.0",
    "request": "~2.30.0",
    "skateboard": "^1.5.1",
    "split": "^0.3.0",
    "weld": "^0.2.2"
  },

The devDependencies property

The devDependencies property of a package.json is almost identical to the dependencies property in terms of structure, with a key difference. The dependencies property is used to define the dependencies that a module needs to run in production. The devDependencies property is usually used to define the dependencies the module needs to run in development.

Inside your package.json, the devDependencies property would look something like this:

    "devDependencies": {
        "escape-html": "^1.0.3",
        "lucene-query-parser": "^1.0.1"
    }

Want to keep going?

If you want to keep learning about Node.js, npm, package.json, and development with the Node.js stack, I've got some awesome articles for you.

We also have a guide on some great utilities for Node.js developers - if you want to kick your developer experience to 11, be sure to check it out to find some tools to help you get there.

The goal with this guide was to help kickstart you with package.json for development with Node.js and npm. If you want to take the leap and ensure that you're always on solid footing with Node.js and npm modules, you should check out NodeSource Certified Modules - an awesome tool to help ensure that you spend more time building applications and less time worrying about modules.