There had been a lot of questions about when db.fsyncLock() is necessary before creating a backup of your MongoDB files† and when it's not.  And there are tricky things around using it, because if you get yourself into a state where you can't fsyncUnlock then you will have to bring down your `mongod` process by force. 

I had been thinking about this problem and realized that one of the tricks to getting the problem minimized would be to run the backup from the same mongo shell sessions/script as the lock/unlock pair, so I wrote a little script to do that.   The script defines a function called 'runBackup' which takes a string representing your backup command (so it could be a shell script, batch file, ec2-snapshot command, tar czf command, anything at all).  The script checks that the server is not already locked, and if it's not, it acquires the lock, runs the command given, and upon its completion runs the unlock command.

The last line shows how to call this function - you can use this as a starting point to create your own.
The code is in my Github repo, along with many other little scripts that might be useful to people who are playing with mongo shell much of the time.

† the general answer is that you only need fsyncLock/Unlock during backup if you are taking a file system snapshot and your journal is on different volume than your data files, or if you are using another way of copying files that can't guarantee getting the data files and journal files as of the same point in time.  No other method requires fsyncLock, not mongodump, nor snapshots that can get data and journal files together.

If you read my posts, you know that I'm a big fan of MongoDB Aggregation Framework.  It was brand new in 2.2 over a year ago and last year in 2.4 release it got faster and picked up some cool new features.   The next production release is 2.6 and aggregation framework continues growing up as a major feature of MongoDB.  Since MongoDB 2.6.0 is now available, I thought I would point out some of the coolest improvements that will make aggregation even simpler and more powerful.

If you want to see the complete list of SERVER tickets fixed for 2.6, it's HERE.  A more readable summary is in the release notes for 2.6.

The biggest and baddest is SERVER-5932, support for aggregations returning a cursor.   Some think that SERVER-3253 is the biggie, that's the one that allows you to redirect the output of of aggregation to another collection (and its cousin SERVER-10097 that allows you the same capability on a sharded cluster).  But if you ask me, getting back a cursor pointing to the results, so I can iterate over it the same way we already do over results from find(), that's the more important one and if I want to, I can then send the results to another target collection (or some other place), so I've been most excited about that.  Needless to say, this makes the old limitation of aggregation returning only 16MB history.

Of course there are many other new features, from major ones or minor ones.  I'm sure you'll be seeing them show up in my future posts as I think of new and novel ways of torturing documents into new shapes, forms and uses!

Sometimes people forget that to measure performance you need to measure throughput (how many requests are you servicing in parallel) and latency (how long is it taking to service each request).  

It boggles my mind when someone runs a single threaded test and sends work to the database serially and then complains about bad throughput.  In order to measure throughput you have to throw as much work at the database server as you can get to it, and usually it means running many clients or multi-threaded clients.

Recently on the MongoDB Google Group someone asked if adding more shards to their cluster would make a particular MapReduce job faster.   Here is how I answered them:

Make sense?

Everyone knows that MongoDB is incredibly easy to get started with.

Most people also agree that MongoDB is very easy to prototype with.  I'm talking especially about the kind of prototyping where you don't really know what the stress points of your application are going to be, what data will be most critical to access fast, what sort of questions you will need to ask of the data after the fact...  In other words, exactly the situation you're in when you first start developing any new application.

MongoDB is easy to install, easy to use with your choice of development languages and it has some wonderful operational features.  Okay, we already know all this, so what is this trap that I am talking about?

I've seen time and time again what happens to a super-fast little MongoDB backed app as the demand on it suddenly skyrockets: if there was no performance/load testing before launch, it's likely that the bottlenecks are a surprise and it's not always clear what they are, where they are, and how to "fix" them.

Here is where people can fall into one of two traps:
  
1. assume that their application load is not a good fit for MongoDB at scale.  
2. assume that MongoDB can handle this if they can figure out how to "tune" it properly.

"Wait a minute," I hear you say.  "Isn't it going to be the case that one of those things is true?"

Absolutely true.  It's possible that the requirements of the application at scale are not a good fit for a document database.  It's also possible that it is the use of MongoDB (schema design, indexing, hardware configuration) that was not done with high scale in mind, and MongoDB is perfectly suited to the workload being thrown at it.

"The trap" is when the assumption about which scenario you're in and reality don't match:

1. Scenario: Assumption is made that application is not a good fit for MongoDB but the application load is actually a perfect fit for MongoDB but poor schema design or bad indexing strategy or suboptimal topology or hardware choices are the cause of bad performance.  
    Result: the team will spend tremendous effort architecting a new system to store their data when they could have improved their MongoDB performance by several orders of magnitude by fixing their design or hardware or cluster topology.

Here are some examples of this:
- There is a missing index.  Solution: you don't fix the slow query by moving the task into a different system, you simply add the missing index.
- Indexes created are not best for the queries/updates running on the system, they take up RAM without providing much of a benefit.  Solution: review and fix your indexes.
- The schema split a number of things into separate collections that should be stored together and every request for the object takes multiple accesses to the database instead of just one.  Solution: reconsider/redesign parts of your schema.
- You have multiple shards but every request is sent to every shard, increasing the overall number of requests that the system must handle.  Sharding scales best when each shard is only required to do work on its portion of the dataset, not the entire dataset.
- You have system that does heavy writes but you chose extremely slow storage system.  Solution: get faster disks (after making sure that your writes aren't unnecessarily inefficient).
- You've made some inappropriate assumptions about how various parts of the system will benefit your use case, or followed outdated or flat-out wrong advice about how to use MongoDB.  Solution: examine your assumptions that don't seem to be holding up, and remove any inefficiency that was created by following the incorrect assumption.

A variant of these scenarios is when an assumption is made about how MongoDB will work with the application and then time is wasted trying to "tune" the wrong thing or worse, "pre"-optimizations are made to cater to some rumored limitation of MongoDB and it's the "optimization" that ends up killing the performance, not the original thing that it was meant to "correct".

2. Scenario: The application load was a terrible fit for MongoDB but for whatever reason an assumption is made that MongoDB will be able to handle it, if only the right "tuning" is made.  
    Result:   The team will spend tremendous effort trying to improve performance of a square peg in a round hole, instead of finding a round peg.

Examples of this include any scenario where you find yourself implementing more database work in your  application than you are asking the database to do for you.   Some examples out in the real world include situations where after migrating from MongoDB to another datastore, the application ended up having a lot less code - that tends to tell me that MongoDB was a poor choice from the start.   

To be honest, I've seen a lot more examples of 1. than 2.  Because MongoDB is quite flexible, it can be a good fit for an extremely wide range of application needs, but if no thought is given to proper schema, indexes and cluster configuration to serve those needs, there is no end to the number of ways that it can fall short of your expectations.

The worst of them all is the assumption that because MongoDB was so easy to install and so easy to get started with (not to mention so fast when it was being run with just some test data) that it will somehow tune itself at scale, without the developer having to give any thought to it.  It would be wonderful if that were true, but at the end of the day, MongoDB is a database, and there is no magic pixie dust that you can sprinkle on it to say "just go faster" - it is my humble opinion, that reports of death of the role of MongoDB DBA have been greatly exaggerated.