MongoDB primary keys are your friend

All documents in a MongoDB collection have a primary key dubbed _id. This field is automatically assigned to a document upon insert, so there’s rarely a need to provide it. What’s interesting about the _id field is that it is time based. That is, the underlying type of _id, which is ObjectId, is a 12-byte BSON type, and 4 of those bytes represent the seconds since Unix epoch.

What’s also special about the _id field is that it is automatically indexed as you can see below by calling getIndexes on any collection.

All MongoDB collections have an _id field as an index
<span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<code class='javascript'><span class='line'><span class="o">></span> <span class="nx">db</span><span class="p">.</span><span class="nx">things</span><span class="p">.</span><span class="nx">getIndexes</span><span class="p">()</span>
</span><span class='line'><span class="p">[</span>
</span><span class='line'>     <span class="p">{</span>
</span><span class='line'>          <span class="s2">"v"</span> <span class="o">:</span> <span class="mi">1</span><span class="p">,</span>
</span><span class='line'>          <span class="s2">"key"</span> <span class="o">:</span> <span class="p">{</span>
</span><span class='line'>               <span class="s2">"_id"</span> <span class="o">:</span> <span class="mi">1</span>
</span><span class='line'>          <span class="p">},</span>
</span><span class='line'>          <span class="s2">"ns"</span> <span class="o">:</span> <span class="s2">"test.things"</span><span class="p">,</span>
</span><span class='line'>          <span class="s2">"name"</span> <span class="o">:</span> <span class="s2">"_id_"</span>
</span><span class='line'>     <span class="p">}</span>
</span><span class='line'><span class="p">]</span>
</span></code>

And as everyone remembers from traditional RDBMSs, indexes are important because they can make document retrieval faster; nevertheless, indexes do consume memory and there is a slight performance penalty when inserting documents as all corresponding indexes must be updated. Thus, while you should seriously consider using indexes, you need to be economical in their usage.

Naturally, searching by a document’s _id is only convenient when you know it. More often than not, documents are searched via other fields and if you find yourself searching via a time series, such as created_at then you are in for a treat.

Imagine a collection dubbed logs that contains simple documents capturing various log messages. A sample document could look like so:

A simple document in a logs collection
<span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<code class='javascript'><span class='line'><span class="p">{</span>
</span><span class='line'>     <span class="s2">"_id"</span> <span class="o">:</span> <span class="nx">ObjectId</span><span class="p">(</span><span class="s2">"51c4ab6d4d6906d494460728"</span><span class="p">),</span>
</span><span class='line'>     <span class="s2">"message"</span> <span class="o">:</span> <span class="s2">"crashed, no such method exception"</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"type"</span> <span class="o">:</span> <span class="s2">"crash"</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"created_at"</span> <span class="o">:</span> <span class="nx">ISODate</span><span class="p">(</span><span class="s2">"2013-06-21T19:37:17.992Z"</span><span class="p">)</span>
</span><span class='line'><span class="p">}</span>
</span></code>

What if I wanted to find all log messages for some date, like today? I could write my query like so:

finding all logs created since June 20th, 2013
<span class='line-number'>1</span>
<code class='javascript'><span class='line'><span class="nx">db</span><span class="p">.</span><span class="nx">logs</span><span class="p">.</span><span class="nx">find</span><span class="p">({</span><span class="nx">created_at</span><span class="o">:</span><span class="p">{</span><span class="s1">'$gt'</span><span class="o">:</span> <span class="k">new</span> <span class="nb">Date</span><span class="p">(</span><span class="mi">2013</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">20</span><span class="p">)}})</span>
</span></code>

If I throw an explain to that query, I can see that because I do not have an index on created_at, a basic cursor is leveraged and all documents in the collection were scanned in order to retrieve my result.

An explain plan attached to my find
<span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<code class='javascript'><span class='line'><span class="o">></span> <span class="nx">db</span><span class="p">.</span><span class="nx">logs</span><span class="p">.</span><span class="nx">find</span><span class="p">({</span><span class="nx">created_at</span><span class="o">:</span><span class="p">{</span><span class="s1">'$gt'</span><span class="o">:</span> <span class="k">new</span> <span class="nb">Date</span><span class="p">(</span><span class="mi">2013</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">20</span><span class="p">)}}).</span><span class="nx">explain</span><span class="p">()</span>
</span><span class='line'><span class="p">{</span>
</span><span class='line'>     <span class="s2">"cursor"</span> <span class="o">:</span> <span class="s2">"BasicCursor"</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"isMultiKey"</span> <span class="o">:</span> <span class="kc">false</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"n"</span> <span class="o">:</span> <span class="mi">2</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"nscannedObjects"</span> <span class="o">:</span> <span class="mi">4</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"nscanned"</span> <span class="o">:</span> <span class="mi">4</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"nscannedObjectsAllPlans"</span> <span class="o">:</span> <span class="mi">4</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"nscannedAllPlans"</span> <span class="o">:</span> <span class="mi">4</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"scanAndOrder"</span> <span class="o">:</span> <span class="kc">false</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"indexOnly"</span> <span class="o">:</span> <span class="kc">false</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"nYields"</span> <span class="o">:</span> <span class="mi">0</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"nChunkSkips"</span> <span class="o">:</span> <span class="mi">0</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"millis"</span> <span class="o">:</span> <span class="mi">0</span><span class="p">,</span>
</span><span class='line'>     <span class="s2">"indexBounds"</span> <span class="o">:</span> <span class="p">{</span>
</span><span class='line'>
</span><span class='line'>     <span class="p">},</span>
</span><span class='line'>     <span class="s2">"server"</span> <span class="o">:</span> <span class="s2">"ghome-computer.home:27017"</span>
</span><span class='line'><span class="p">}</span>
</span></code>

As you can see, searching via the created_at field can be inefficient; thus, you might be tempted to throw an index on that field. This would naturally make that particular query more efficient, however, you would incur the cost of a new index which is more memory consumed and inserts would be slightly slower due to an update to that newly created index.

As it turns out, because the _id field embeds Unix epoch in it, you can just as easily craft a find expression without including the created_at field. For example, the MongoDB Ruby driver allows you to create ObjectId’s from a Time like so:

Related:
1 2 Page 1
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.