The MongoDB Aggregation Pipeline by Example

MongoDB's aggregation framework transforms and summarizes documents through a sequence of stages. Each stage takes a stream of documents, does some work, and passes results to the next stage. Understanding five or six core stages covers the vast majority of real-world reporting needs.

The Pipeline Mental Model

Think of the pipeline as a Unix pipe:

bash
db.orders.aggregate([
  { $match: ... },     # filter — like WHERE
  { $group: ... },     # aggregate — like GROUP BY
  { $sort: ... },      # order — like ORDER BY
  { $limit: 10 }       # truncate — like LIMIT
])

Stages execute in order. The output of one stage is the input to the next. You can repeat stages — two $match stages in sequence is valid (though usually you combine them for clarity).

$match: Filter Early

Always put $match as early as possible. A $match at the start of the pipeline can use an index; one further down cannot. Filtering early also reduces the number of documents flowing through expensive later stages.

js
// Find orders from the last 30 days with status "shipped"
db.orders.aggregate([
  {
    $match: {
      status: "shipped",
      createdAt: { $gte: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000) }
    }
  }
])

$group: Aggregating Data

$group collapses documents by a key and applies accumulators:

js
// Total revenue per customer
db.orders.aggregate([
  { $match: { status: "shipped" } },
  {
    $group: {
      _id: "$userId",
      totalRevenue: { $sum: "$total" },
      orderCount:   { $sum: 1 },
      avgOrder:     { $avg: "$total" }
    }
  },
  { $sort: { totalRevenue: -1 } },
  { $limit: 10 }
])

The _id field is the grouping key. Setting _id: null aggregates all documents into a single result — useful for computing a collection-wide total.

Common accumulators: $sum, $avg, $min, $max, $push (collect values into an array), $addToSet (unique values), $first, $last.

$lookup: Joining Collections

$lookup performs a left outer join from one collection to another. It is the aggregation equivalent of a SQL JOIN:

js
// Attach user data to each order
db.orders.aggregate([
  { $match: { status: "shipped" } },
  {
    $lookup: {
      from: "users",
      localField: "userId",
      foreignField: "_id",
      as: "user"
    }
  },
  // $lookup always produces an array; unwind to a single document
  { $unwind: { path: "$user", preserveNullAndEmpty: false } },
  {
    $project: {
      orderId: "$_id",
      total: 1,
      userName: "$user.name",
      userEmail: "$user.email"
    }
  }
])

A few things to note:

• $lookup produces an array field even when there is exactly one match. $unwind flattens it.
• $lookup cannot use an index on the foreign collection in all configurations; always verify with explain(). See the MongoDB indexing guide to understand what the query planner is doing.
• For high-volume pipelines, an uncorrelated $lookup using a pipeline sub-stage can be more efficient than the simple localField/foreignField form.

$unwind: Deconstructing Arrays

$unwind turns a document with an array field into one document per array element:

js
// Count how many times each tag appears across all posts
db.posts.aggregate([
  { $unwind: "$tags" },
  {
    $group: {
      _id: "$tags",
      count: { $sum: 1 }
    }
  },
  { $sort: { count: -1 } }
])

Without $unwind, the tags array would be treated as a single value by $group. After $unwind, each tag becomes a separate document and can be grouped independently.

$project and $addFields: Shaping Output

$project controls which fields appear in the output and can compute derived fields:

js
db.orders.aggregate([
  {
    $project: {
      _id: 0,
      orderId: "$_id",
      total: 1,
      taxAmount: { $multiply: ["$total", 0.08] },
      yearMonth: { $dateToString: { format: "%Y-%m", date: "$createdAt" } }
    }
  }
])

$addFields is like $project but keeps all existing fields and adds new ones — useful when you want to annotate documents without re-listing every field.

$facet: Running Multiple Pipelines in One Pass

$facet runs several sub-pipelines on the same input documents simultaneously, which is ideal for paginated list APIs that need both a page of results and a total count:

js
db.products.aggregate([
  { $match: { category: "electronics" } },
  {
    $facet: {
      results: [
        { $sort: { price: 1 } },
        { $skip: 0 },
        { $limit: 20 }
      ],
      totalCount: [
        { $count: "count" }
      ]
    }
  }
])
// Returns: { results: [...], totalCount: [{ count: 142 }] }

This replaces the common anti-pattern of running two separate queries — one for the page of data and one for the total — which doubles your database round trips. This is a key technique for implementing cursor and offset pagination efficiently.

Performance Tips

1. Match early. Put $match before any $lookup, $unwind, or $group.
2. Project early. After $match, drop fields you do not need with $project to reduce document size flowing through later stages.
3. Index on $match and $sort fields. The first $match and the first $sort in a pipeline can use indexes. Later stages cannot.
4. Use allowDiskUse for large $group operations that exceed the 100 MB in-memory limit: .aggregate([...], { allowDiskUse: true }).
5. Check explain("executionStats") on the full pipeline, not just individual queries.

Data modeling decisions also affect pipeline performance — knowing whether to embed or reference changes how many $lookup stages you need. For the server-side Node.js code that runs these pipelines, error handling patterns ensure pipeline failures are surfaced cleanly rather than silently returning empty results.

Conclusion

$match, $group, $lookup, $unwind, $project, and $facet cover most real reporting requirements. The pattern is always the same: filter early, join what you need, aggregate, shape the output. Validate your pipeline performance with explain() and profile before moving complex reporting logic into application code.