Going Deep on Mongoose Deep Population, and How To Populate Across Multiple Levels

If you've made the switch from SQL to NoSQL, you've probably started to dabble into Mongoose population as a means to make up for the lack of table joins.  This, unfortunately, is one area where NoSQL databases fall short of their SQL counterpart.  Mongoose population can be tricky to setup.  This article take a deeper dive into Mongoose populate, to help avoid some pitfalls that you may encounter setting this up.

What Mongoose Populate Does

Mongoose stores data sets in JavaScript objects, hence you may have a JS object for Users.  A user in turn, may have friends, which are an array of other Users.  They will also likely have arrays of other objects, such as Posts, Comments, and even Comments of Comments.

In SQL you could simply perform table joins.  For NoSQL databases like Mongo, your best option is to create separate JS object schemas for each, and reference the connection by object ID.  For example, the User object will contain arrays for other Users, Posts, Comments, etc.  Each array would only contain the object IDs of the objects they are pointing to (i.e. User IDs, Post IDs, Comment IDs, etc).  When querying data, you can then call the populate function to load the data of the associated object for the arrays that you select.

Yes, this has drawbacks.  For example, if you delete an object such as a User, you will also need to find everywhere that references that user, and delete the user ID from there as well (such as a user's friends).  Not doing so can cause issues when trying to populate the ID's that no longer exist.  Even with these drawbacks, you'll find that working with the populate function is still probably your best option, over other JavaScript hacks.

Populating Across Multiple Levels

Mongoose provides great documentation for how to get started with populating arrays.  Below is a simple example they have provided for populating across multiple levels.  In this case we have Users, with an array of friends that are other Users.  We would like to populate the object to show the user's friends, and friends of friends.

First, we need the User schema:

const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const ObjectId = mongoose.Schema.Types.ObjectId;


const userSchema = new Schema({
      name: String,
      friends: [{ type: ObjectId, ref: 'User' }]
});


const User = mongoose.model('User', userSchema);

Notice, we are only tracking an array of Object IDs for the user's friends.  Those Object IDs are just other User IDs.

Next, we'll query by user name, and populate friends, and friends of friends:

User.findOne({
    name: 'Val'
}).
populate({
    path: 'friends',
    // Get friends of friends - populate the 'friends' array for every friend
    populate: { path: 'friends' }
}).
exec(function (err, user) {
    if (err) return handleError(err);
    console.log('Here is the populated user: ', user);
});

The return value should convert all of the Object IDs in the friends array, to the actual data in the associated User schema.  In the above example, we've done this for 2 levels, so it will also populate data for friends of friends.

Populating With Multiple Schemas Across Multiple Levels

In many applications, you'll want to do a lot more populating.  Here's how to do that. Let's add posts and post comments to our object.

First off, let's define our Schemas. These will need to be readily available:

const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const ObjectId = mongoose.Schema.Types.ObjectId;


const userSchema = new Schema({
    name: String,
    friends: [{ type: ObjectId, ref: 'User' }],
    posts: [{ type: ObjectId, ref: 'Post' }]
});
const postSchema = new Schema({
    name: String,
    content: String,
    comments: [{ type: ObjectId, ref: 'Comment' }]
});
const commentSchema = new Schema({
    name: String,
    content: String
});


const User = mongoose.model('User', userSchema);
const Post = mongoose.model('Post', postSchema);
const Comment = mongoose.model('Comment', commentSchema);

Next, we query and populate the document.  Let's say we're no longer interested in friends of friends, but we want to populate friends, posts, and post comments.  Here's how we populate these:

User.findOne({
    name: 'Val'
}).
populate({
    path: 'friends',
    model: 'User'
}).
populate({
    path: 'posts',
    model: 'Post',
    populate: {
        path: 'comments',
        model: 'Comment'
    }
}).
exec(function (err, user) {
    if (err) return handleError(err);
    console.log('Here is the populated user: ', user);
});

Notice, how we now have defined the model in each object.  Mongoose will often need this with multiple schemas.  

To populate multiple paths, we simply repeat the populate function.  To dig deeper, we just populate a level deeper within the object.  

Final note, if you get stuck, be sure to log out your errors on execution.  Mongoose provides good error handling for those who listen!