Virtual Populating Embedded Docs with Dynamic "ref"

[jimmytsao]

Do you want to request a feature or report a bug?
This is a request for clarification on usage or feature if it's currently not possible

What is the current behavior?
Unable / unclear on how to perform a virtual population of embedded docs with dynamic "ref"

If the current behavior is a bug, please provide the steps to reproduce.

const mongoose = require('mongoose')
mongoose.connect('mongodb://localhost/mongoosetest')
mongoose.connection.once('open', () => {
  const Schema = mongoose.Schema

  // Generate Book Model
  const bookSchema = new Schema({ productId: Number, name: String })
  const BookModel = mongoose.model('Books', bookSchema, 'books')

  // Generate Magazine Model
  const magazineSchema = new Schema({ productId: Number, name: String })
  const MagazineModel = mongoose.model('Magazines', magazineSchema, 'magazines')

  // Purchases Schema
  const purchasesSchema = new Schema({
    refKey: String, // This will be either "Books" or "Magazines"
    productId: Number
  })

  // User Schema 
  const userSchema = new Schema({
    name: String,
    purchases: [ purchasesSchema ]
  },
  {
    toJSON: { virtuals: true },
    toObject: { virtuals: true }
  })

  // Add virtual for population of embedded field with dynamic ref
  userSchema.virtual('purchases_$', {
    ref: doc => {
      console.log('DOC', JSON.stringify(doc, null, 2))
      /*
        DOC "{
          "_id": "5ad000068fc2370c7713c4bd",
          "name": "John Doe",
          "purchases": [
            {
              "_id": "5ad000068fc2370c7713c4bf",
              "productId": 11,
              "refKey": "Books"
            },
            {
              "_id": "5ad000068fc2370c7713c4be",
              "productId": 22,
              "refKey": "Magazines"
            }
          ],
          "__v": 0
        }"

        The full document is provided here but how do I target
        the "refKey" field of each embedded document in the purchases
        array?
      */
      return doc.purchases.refKey
    },
    localField: 'purchases.productId',
    foreignField: 'productId'
  })

  // Create User Model
  const UserModel = mongoose.model('Users', userSchema, 'users')

  // Generate Items
  const book = { productId: 11, name: 'Harry Potter' }
  const magazine = { productId: 22, name: 'Time' }
  const user = {
    name: 'John Doe',
    purchases: [
      { productId: 11, refKey: 'Books' },
      { productId: 22, refKey: 'Magazines' }
    ]
  }

  Promise.all([UserModel.create(user), MagazineModel.create(magazine), BookModel.create(book)])
    .then(function () {
      UserModel
        .findOne({ name: 'John Doe' })
        .populate('purchases_$')
        .lean()
        .then(results =>
          console.log(JSON.stringify(results, null, 2))

          /*
            {
              "_id": "5ad000068fc2370c7713c4bd",
              "name": "John Doe",
              "purchases": [
                {
                  "_id": "5ad000068fc2370c7713c4bf",
                  "productId": 11,
                  "refKey": "Books"
                },
                {
                  "_id": "5ad000068fc2370c7713c4be",
                  "productId": 22,
                  "refKey": "Magazines"
                }
              ],
              "__v": 0
            }
          */
        )
    })
})

What is the expected behavior?
I was expecting the virtualpurchases_$ to be populated with "Harry Potter" and "Time" documents

          /*
            {
              "_id": "5ad000068fc2370c7713c4bd",
              "name": "John Doe",
              "purchases": [
                {
                  "_id": "5ad000068fc2370c7713c4bf",
                  "productId": 11,
                  "refKey": "Books"
                },
                {
                  "_id": "5ad000068fc2370c7713c4be",
                  "productId": 22,
                  "refKey": "Magazines"
                }
              ],
              "purchases_$": [
                {
                  "_id": "5ad000068fc2370c7713c4ba",
                  "productId": 11,
                  "name": "Harry Potter"
                },
                {
                  "_id": "5ad000068fc2370c7713c4bb",
                  "productId": 22,
                  "name": "Time"
                }
              ],
              "__v": 0
            }
          */

Please let me know if this is the correct to populate or if there is a different way that you would recommend.

Thanks!

Please mention your node.js, mongoose and MongoDB version.
node: 9.7.1
mongoose: 5.0.14
mongo: 3.2.5

[vkarpov15]

Currently, the best way to do this is just return an array of the refKeys:

  // Add virtual for population of embedded field with dynamic ref
  userSchema.virtual('purchases_$', {
    ref: doc => {
      return doc.purchases.map(purchase => purchase.refKey)
    },
    localField: 'purchases.productId',
    foreignField: 'productId'
  })

There's room for improvement here on mongoose's end because you'll get unpredictable behavior if you have a book and a magazine with the same id. Below is the output with mongoose.set('debug', true); and the above schema config:

Mongoose: magazines.find({ productId: { '$in': [ 11, 22 ] } }, { fields: {} })
Mongoose: books.find({ productId: { '$in': [ 11, 22 ] } }, { fields: {} })
[ { _id: 5ad8c35a4eee4f2a955be68e,
    productId: 11,
    name: 'Harry Potter',
    __v: 0 },
  { _id: 5ad8c35a4eee4f2a955be68d,
    productId: 22,
    name: 'Time',
    __v: 0 } ]

It'll search for both ids in both collections, but as long as you don't have id overlap the only issue with this is a trivial performance overhead. Does this solution work for you?

[jimmytsao]

Yes - the ids in both collection will be unique in my scenario and this solution works. Thanks for the clarification!

[mxmaxime]

Hello,

Thanks, it worked for me too!