Welcome back to another tale from the trenches of Customer Success! As we roll into the summer months in the Northern Hemisphere, we turn our thoughts to snapshots. Snapshots are one of the essentials of any storage system, giving the administrator essentially a read-only in-time recovery point for their file system. It must be stressed that snapshots should never be confused with backups. In more than 20 years of doing this, the number of customers I’ve encountered who rely solely on snapshots as their backup methodology is frighteningly high. Snapshots should complement your backup policy, but never replace it.
If you’re new to storage, you might be asking yourself, “what are snapshots?” Snapshots are best described as a point in time picture of the active file system. When a snapshot is taken, the active file system is frozen, and that becomes your recovery point. Writes will continue to flow into the system, and if a file is opened that was created prior to the snapshot taking place, it will simply be referenced from within the snapshot itself. All activity continues as normal.
Where things get interesting is if I edit or delete a file that’s been locked by a snapshot. For example, I have the following text written in a file –
Hello world!
I then take a snapshot. Awesome, no real space utilization short of the reference block saying “hey, there’s a snapshot.”
I now go and delete that file from my storage array. Since it’s locked by the snapshot, its space usage is now reflected in the snapshot rather than the active file system. For a simple file like the text written above, this space usage is negligible. However if you delete say a 10TB directory, suddenly that 10TB usage is going to be reflected in the snapshot capacity, leaving you with a net change of zero, as all you’ve done is basically moved blocks from one accounting bucket to another. A more in-depth (and technical sounding) study can be found on the Qumulo documentation portal under How Snapshots Work.
Qumulo also allows for the creation of snapshot policies. These policies give the storage administrator the ability to automatically create recovery points, and then allow end users to recover the data on their own. Policies also provide for the granularity that admins are accustomed to – having hourly, daily, weekly and monthly recovery points.
One question that confuses customers on a regular basis is how much space a snapshot actually uses. If you only have a single snapshot, it’s not a problem, but things get complicated as you add more snapshots and start adding and deleting data. This is known in the circles I frequent as regular day to day operations. The crux of the issue is due to the nature of snapshots themselves. Since they are read-only points in time, they are immutable – any changes will have to be reflected somewhere else.
Take our example above:
Hello world!We take a snapshot and our very cordial message is locked in place. Great! We’d hate to lose that. An hour later, we change it up to:
Howdy world!Makes it a little more down home and gives it a country feel. We take another snapshot.
At this point, I have snapshot 1 that references it as ‘Hello world!’ and snapshot 2 that references it as ‘Howdy world!’. To the file system, there is a delta there of about four letters it needs to track in some manner or another.
Time passes, and I decide that’s too friendly, so I just delete the file as a whole. The file now only exists in snapshot form, so my space referenced moves from the active filesystem to snapshots 1 and 2. If I were to delete snapshot 2, I’m not really freeing up that much data, as snapshot 1 will still exist. Likewise, if I delete snapshot 1 then snapshot 2 would simply grow in size to reflect how the file should look when snapshot 2 was taken.
Due to the reconciliation of data, it can make it difficult to know how much space will be recovered if a snapshot is deleted. The best rule of thumb I can offer up is to always go after the oldest snapshots on the system first if you’re looking to clear space. That way you can ensure things will be freed up if possible, instead of just rolling into another data bucket.
That’s a wrap. Next time we’re going to poke at some new features Qumulo released to get forklifts out of your data center. Until then, amigos!
