Time Machine and SmartGuard

Yesterday Time Machine saved my life once again even though i strongly suspect it also was the culprit of the crash. I mean, what happens when Time Machine backs up my pvm file while working in Windows...? The last six months i've had the "APFS fsroot tree is invalid" problem twice and the pvm file is to blame. I believe this has to do with Time Machine backing up the pvm file while working in Windows. The first time i formatted my hard drive and started all over again but this time i restored the pvm file after hours of research. Voila, everything works great again. Because of this i added "~/Parallels" to the list of folders Time Machine should exclude. Still i want my virtual machine backed up so my question is about SmartGuard... I have now set the SmartGuard on and ticked the "Optimize for Time Machine" box but the Parallels folder is still excluded from Time Machine. Do i have to include the Parallels folder in Time Machine or are snapshots stored elsewhere? Also i would like to know if someone else have come to the same conclusion as i have?

 

Hi, @tomfri you can just enable SmartGuard as you did. For more info please check https://kb.parallels.com/8827
Also if you decide to backup your VM please shut it down and plz check this KB for more info about VM's backup.

 

tomfri raises some good points and, having had Windows destroy my user account two days ago, this topic has a lot of meaning for me.
The links provide some cursory information but I haven't found a resource that provides detail.
It appears that the purpose of SmartGuard is to create snapshots. Per the UI, the number of snapshots and the frequency with which a snapshot can be created is controlled or the user can check the "Optimize for Time Machine". If I understand correctly, the SmartGuard feature deals only with creating snapshots. Is that correct?
If the PVM is backed up by TimeMachine, TM will create a backup of the entire PVM and will then, assuming that SmartGuard is enabled, will then backup just the snapshots. Is that correct?
The docs clearly state the PVM should be shut down when doing a backup. That's understandable and, with backup software that could be set to run at "off hours", easily accomplished. An issue arises, perhaps, when using Time Machine because it does an hourly backup. The technical solution is to shutdown my VM's every hour to wait for TM to run but no one could argue that to be a practical solution and keep a straight face. Given that, I have to think that the "shut it down" advice does not apply when using Time Machine or, perhaps, if you're using Smart Guard. Is that correct?
Maria, could you be so good as to comment on the questions that I've raised?
Perhaps I'm trying to make too fine a point here but I think everyone would be better served if more information on SmartGuard and backups/Time Machine was available.

 

Forgive me for still being confused. It would be helpful for Parallels to make a youtube video explaining the process of setup and basic explanation of how it works and integrates with Time Machine, both for backup and for restore. It seems that recommended article 8827 is out of date - only mentions v. 13. I see contradictory points in these forums and want to spell it out once again to make sure I have it right and maybe even understand the process. I think I am to include my windows .pvm in the Time Machine backup and I am to check the SmartGuard option in the settings. I gather that Time Machine backups up Parallels but only backs up the small part of it that is the snapshots.
If I were to restore my .pvm from Time Machine, how would that work? Normally you navigate to where the file/application is stored on the Mac, but if that .pvm file is not backed up, then how would I proceed? What if reinstalled a hard drive and reinstalled Windows: would my backup work there?
I use Parallels v. 14 with Windows 10.
Thanks
Jim

 

The other thing is that if dvonroeder is correct and parallels warns against making a backup while the virtual machine is running, then that essentially means that you can't do automatic Time Machine backups. If Parallels Team could confirm about this point in regards to the danger of the hourly TM backups.

 

I heartily concur with the recommendation about creating a video. I wouldn't go so far to say that the hourly Time Machine backup can't be used (I suspect that Parallels is being very conservative) but I certainly don't want to find out the hard way. My approach was to enable SmartGuard and to ensure that Time Machine captured the initial file when Parallels was not running. Smart Guard blocks activity while it's working so it's able to create a valid delta file in the Parallels folder. Time Machine picks up that file when it does a backup.
This topic is not complex but it's not well documented and, second, the idea that you can exclude your pvm from Time Machine and still be able to recover your pvm file just using the SmartGuard files doesn't make sense to me.
Parallels users would be well served by up-to-date information on this topic.

 

For a few years, I've been using Carbon Copy Cloner to backup the PVMs on a daily or 4 times daily schedule. The VMs are never shut down and the backup PVMs have been fine when I've needed to use the backups.
I want to integrate this routine with Time Machine so that only incremental backups are required - rather than the entire PVM which takes up time and CPU resources. Hence, I would also be interested in a detailed reply from Parallels regarding the mechanism of Time Machine backups.

 

Greetings,
I just found this thread. I'm very curious to know about your "APFS fsroot tree is invalid" issue.
My ongoing issue is I (and others) have variable size VM files, and when we try to shrink it, the entire Mac OS hangs up and the only way to get it back is to power down. This corrupts the Mac file system as well as the Windows VM. The only thing to do is restore the entire system from Time Machine as well as restoring the VM from backup. Please see this discussion: "Mac host file system corruption during shrinking of PD13 Win10 VM". Your issue sounds familiar. So far I have had a little communication from "Maria" but no relief. I have given up on shrinking the VM file. As far as the VM and Time Machine are concerned, the only way I back it up is to shut down the VM and manually copy the VM file to external storage. I do not trust SmartGuard at all based on what I've been through.

 

@dvonroeder has asked some VERY valid and interesting questions that I would also like to have the answer to.

Could someone from the Parallels team respond to them? This would be very much appreciated by many of us.

 

Totally agree, there is no direction from the Parallels team as to how to correctly set up Smart Guard. This is unclear as to if it should be set to exclude back ups or not once Smart Guard is enabled. https://kb.parallels.com/8827

I have it working now, and I *believe* I have it set up correctly, and thought i'd detail my use case. Time Machine was backing up the entire VM with every single back up -50 GB each hour, and it chewed through 2TB of my 8 TB Time Machine back up drive in a a week.

To fix this, I successfully enabled Smart Guard. Then quit Parallels and forced Time Machine to "back up now". The first time after enabling, TM did another "full" VM back up of 50GB. But once that first one was done, I forced it to do another "back up now" and this one was 75MB and super fast. Each Time Machine back up after that is also just a super fast incremental back up. You can verify that it's working by "entering" Time Machine and navigating to all the VM files.

In terms of a traditional back up (not Time Machine) that is the one that needs to be done with the VM closed. I use Super Dupe (on a separate external HD) and have it auto run in the middle of the night when I have the VM shut down. Seems to be working fine.

I *no not* have the VM excluded from Time Machine in the Parallels preferences. My understanding is that it must be included in order for Smart Guard to work - something that is unclear from the docs.

 

Thanks for sharing this in detail - you are indeed correct about how unclear it seems to be. Am encouraged to hear how you've managed to get your TM backups down to seemingly increments.

Wanted to know though if you've tested the restore successfully? How do you rejoin the baseline VM backup file with its subsequent increments? Doesn't TM eventually delete the baseline file? Also, I wasn't clear why you were using SuperDuper as well to create a full copy of your VM when you have the TM backup set up already?

 

Last post here was March 17. It seems Parallels never did sign in to answer questions. Is there any new information.
The biggest issue seems to be the inability to keep the Windows VM running, while Smart Guard is Enabled, and Time Machine is doing it's Nightly backups. I monitor via Terminal all the TM activity, and it is definitely every hour at least at first. So it's crazy not be able to keep the VM running.
Was this ever resolved?? It sounds to me like it's worth getting Carbon Copy Cloner, which has support, etc...and somehow doesn't require the VM to be shutdown during backups.
???

 

Parallels Team,
Please provide guidance (clear step by step examples) of how your engineers envision (and, since backup features were published there was a vision) and successfully tested backing up and recovery from the same for Windows 10 VM/Parallels itself on (some) Mac OS X - (in short, the entire bootable partition) in a manner that involves relative ease of backup and recovery.
Why does ease matter? Because anyone who has ever worked in IT knows from experience
that backup methods that are not fairly painless go unused by a double digit percentage of all
users.
- How are internal power users doing it - ideally with minimal complexity and VM down time?
- How do you rejoin the baseline VM backup file with its subsequent increments?
- Does VM need to be included in TM backups for SmartGuard to work? How was this handled
in UAT? Someone knows.

To remain silent on matters this important, especially when your own documentation is vague,
hurts your brand more than if you share that Windows (and/or other?) VMs cannot be captured
while running.
If SmartGuard has limitations over some third party image or backup solutions then please admit it,
or share clear directions on best practices optimization, and let users be the judge of whether or not that is adequate for them (once able to consider tradeoffs).

- What are real-world external approaches you see working well? Are they hybrid approaches with third party utilities as well?
- How is TM configured in those instances - if at all?

Some of us with strong technical skills have spent many hours researching forums and feel as lost
as when we started.
Sharing rather than hiding these points may give people more confidence in your
brand. VM software doesn't have to do everything. But it does have to allow software to be able to be
backed up in some clear and practical way, for people to risk using it over your competitior's products.
This is too mission critical and time consuming to leave us guessing, asking one another and experimenting. A "machine" virtual or otherwise, must be reliable in this day and age. Consider
a proven process to make it so part of a MVP (minimum viable product/solution), and that options with CLARITY in documentation and forums will not only add revenue but reduce your contact center/support costs.

So be a hero, and raise it with your heads of product and operations (if even you do not know how to reply here!!) to do what is ultimately in the best interests of your company. Or, risk consumers on the fence deciding to go with HyperOS or VMware Fusion because your silence is ridiculous, considering what is at stake for them.






What is the workaround for the inability to keep the Windows VM running, while Smart Guard is enabled, and TimeMachine is running hourly?

 

What is the workaround for the inability to keep the Windows VM running, while Smart Guard is enabled, and TimeMachine is running hourly?

Is there anyone from the Parallels tech or product side who monitors your own forum? Even a response that "there is nothing planned" - is some answer, showing some respect for your customers' concerns and our efforts to reach out to and seak answers from those closest to the product, no?

 

Hi, could you please explain what kind of workaround would you like to use and how would you like to use it?

Also unfortunately we cannot share any plans or new features before it has been released.

 

Hi all. I know the thread is kind of old, but I just wanted to share with you guys my response in another thread that may shed some light on the situation. No, I do not mention anything about problems arising due to the backups and/or the use of SmartGuard, simply because I have never experienced them. Also, please note that in the thread where I posted this before, the original poster had a question related to using the client's own encryption. Hence the part where I talk about the encryption. Here you are. I hope this helps.

"
...myself, having struggled with the pairing of TimeMachine and Parallel's SmartGuard snapshots, and the very poor documentation on the subject, decided to share what I discovered. Basically I needed to truly understand how the snapshots worked, and if I could rely on them in case I ever needed to restore a VM from a snapshot using TimeMachine (and the snapshots). In other words, I had the same "Sub-question" that you had, and then some.
So, here it goes...

First, disclaimer: I do NOT work for Parallels and I am in no way, shape, or form the final authority on the software. I did do extensive testing though, and the results of all my testing, which involved going through several iterations of waiting for snapshots to be automatically created, and test recovering the VM several times using TimeMachine, at different stages (with no snapshot yet taken, with 1 snapshot taken, with two, and more snapshots taken, with both the VM powered on and off), AND, yes, my TimeMachine drive IS encrypted.

I'll first give you the bottom line of all my testing, and then I will expand.

Bottom line: Yes. It all works. Perfectly. Seamlessly.

Now, you're thinking "That does not answer any of my questions!" But it does. In an Apple philosophy kind of way... As in, Apple (and in this case Parallels) says you do not need to know, so you do not need to know. Of course, we The Users disagree. We DO need to know.

So, here's my expanded answer:

1- I will put this one first, because you mentioned you are doing the encryption in the guest. That presents the problems you mentioned. You actually can keep on doing that. But, if you really want your data and your backups safe, and you want the final results that you described, you need to (also) encrypt your TimeMachine backup drive itself. Yes, I am saying that if you want to have your VM encrypted at the guest level, and you want your backup safely encrypted when the VM is powered on, you must encrypt your TimeMachine drive backup as well (host level). If you do that, it really does not matter if the VM is powered on or not. The backup will be encrypted on the backup drive (TimeMachine), and you (or anyone else) will only be able to restore the VM to that specific powered on state (and access any sensitive data) if they have the decryption password of the encrypted TimeMachine backup drive. Makes sense? (Yes, I know you will be encrypting twice here. But that's the price you pay for what you want to achieve.)
2- Number one pretty much takes care of your encryption situation. Now on to how SmartGuard Snapshots actually do work really well with TimeMachine... I thoroughly tested each and every single scenario I mention here:
a) If there aren't yet any SmartGuard snapshots of the VM: TimeMachine backs up the entire .pvm file, however large it is, regardless of whether the VM is powered on or not. Again, if you have the backup drive encrypted, you are safe, even with the VM powered on.
b) If there is already 1 SmartGuard snapshot, AND TimeMachine has at least one full backup of the entire .pvm file, THEN it keeps the full .pvm backup, does not back it up again, and backs up the snapshot.
c) If there is already 1 SmartGuard snapshot, BUT TimeMachine never had a chance to backup the entire .pvm, THEN it backs up the full .pvm, AND then it backs up the snapshot.
d) And the pattern repeats itself... If there are 2 or more SmartGuard snapshots, TimeMachine will back up the full .pvm file only if it wasn't backed up before, and it will do the same for any SmartGuard snapshots that it hasn't yet backed up. That is, if the snapshot has already been backed up before, it is already backed up, and it is not backed up again. But any new snapshots that weren't yet backed up WILL be backed up. However many (non-backed up) snapshots there are.

If you follow my reasoning 1 thru 2, a thru d, you see that everything gets backed up, quite efficiently, without data duplication.

Now you may ask (I did!): "Ok. How does the restore work?" That was my main question, because I wanted to make sure I could rely on SmartGuard and TimeMachine configured this way. It is quite simple, actually. When/If you want to restore, you simply enter TimeMachine, browse to the folder that contains the .pvm file, and click "Restore". TimeMachine/Parallels SmartGuard will automatically calculate which SmartGuard Snapshot was the closest one taken prior to the TimeMachine backup and restore the main first full .pvm backup, and then apply the latest SmartGuard Snapshot, which will bring your machine to the closest state it was in, when the last snapshot was taken, before the last backup was taken, for the date that you choose.

Practical example:
1- TimeMachine runs and backs up your entire .pvm, say, 01/01/2030.
2- SmartGuard takes a snapshot 01/02/2030.
3- TimeMachine runs and backs up the 01/02/2030 snapshot on 01/03/2030.
4- SmartGuard takes a snapshot 01/04/2030.
5- TimeMachine runs and backs up the 01/04/2030 snapshot on 01/05/2030.
5- SmartGuard takes a snapshot 01/06/2030.
6- At this point, on 01/07/2030, you experience a catastrophic failure of your drive and you lose the .pvm completely.
7- After resolving the problem with the drive, you enter TimeMachine, browse to the folder where the .pvm file resides. Go back in time to 01/06/2030, and you click restore. TimeMachine knows it has a good full backup of the .pvm from the first original backup, and it also knows that the latest available backed up SmartGuard Snapshot for the date you chose (01/06/2030) is from 01/04/2030. It restores the .pvm file exactly as it was as of 01/04/2030, which is the latest good backup/SmartGuard Snapshot combination per the restore date chosen.

I know I typed a book here. (And I am not the best writer ) But I hope I explained everything.

Best, Raphael
"