I'm having this issue too. I've tried both OpenVDB and Point Cache. Low Divisions and High Divisions. External Path and Lib Path. In the past, I've had issues where it wouldn't bake all the way and I would have to run it again. Usually the cause was being on a frame other than the first frame of the bake, which seems odd. If I went back to frame one and started the bake again, it would usually work the second time and I'd never have any issues.

I've attached the most recent project that has has the bake files vanish. This was sent to me by a guy helping to make the smoke assets we need for some VFX

here is a link to a dramatically simplified .blend with only 5 smoke domains. (Imagine 4-5 more smoke domains on 10 more layers and that is my scene for the torches). Bake these domains using the external settings, only maybe changing the folders, and then save the project. Close it. Restart the computer or just wait awhile and then make move the playhead to some different frames, and eventually the bake files will disappear. I dont even know what's causing this (thats the problem), or else I would give more specific steps to reproduce it, but again, the behaviour is consistent across 3 machines and different .blend projects on different platforms (mac & windows). We REALLY need some help with this. Cant get any bakes to stay put... and with large bakes (over 200gb collectively and 12+ hours into bakes) this is a HUGE HUGE problem.
Bake sample can be downloaded here: bake_sample.zip

More files being deleted. Started a smoke bake that ranged from frame 1-500.

Did a bake with 5 new domains a couple days ago for torches. Started at frame 1. Baked the domains. Then moved the playhead to a half-way point through the animation to check the fire and they were all there. Then I did a F12 render and 2 of the torches were missing. Checked the bake files and the files for those two domains were gone. Guys, this happens over and over and over again. It’s killing me. I’m literally not able to do anything that requires a bake. We are losing bake files at work that take days to bake! Please help us find a fix or a work-around or something.

Over the three weeks I think Ive managed to eliminate the deleting of bake files. I did this by making all the bake files READ ONLY after they were done baking. Today I viewed the process output from one render job and I noticed a ton of errors where blender was reporting that it could not delete a file. Could it be, that blender is trying to delete my bake files, but cant because I made them read-only??

Hi Mike,
can't access your sample file -- but I kind of know that problem. I encounter it as unfortunate combination of circumstances.

As a starting point: whenever a simulation is not yet marked as baked in blender, then on any move of the playhead, Blender seemingly marks its internal cache as dirty; and when this especially touches some smoke sim, it attempts to calculate the current frame. Now, there is a special twist: a smoke sim can only be calculated, when you enter it from the first frame, since it needs to be calculated consecutively. For that reason, blender discards any "stale" cache data behind the current position, which also results in deleting the disk cache.

The only way to prevent that from happening is to have the smoke sim marked as "baked". You can see this in the property pane of the smoke domain, on physics tab: when baked, most settings are grayed (can't be edited), and there is a "free bake" button in the cache area. (you can also see this state when you look at the data blocks of the blend file)

Usually, if you just bake interactively, by pressing the "bake" button, blender freezes during the bake process, and when successfully done, immediately marks this simulation as baked. Then you must save the file to capture this state. If you just close, the files are in the cache, but next time blender considers them as stale, since it doesn't recognise the simulation as baked.

And now comes the twist: if you bake through a python script, e.g. on some external machine with many cores, then this script must actively save the blend file and you need to retrieve not only the baked frames, but also the blend file.

There is a second workaround (and this is what I am using in practice): Once you have calculated the simulation on said external machine, you retrieve the cache frames and place them into another directory (not the blendcache_* directory). Make sure the playhead is somewhere in the middle of the smoke sim's area. (It should not show any smoke at that point, since the internal cache is still stale). Then you toggle the "external" checkbox below the cache settings and enter the path to the external directory. Important: now wait until the smoke sim shows up in the 3D viewport. Never move the playhead, don't touch anything. And when it has showed up, which requires a considerable amount of time where your CPU goes 100%, immediately click the "current cache to bake" button. And then immediately save the blend file. This gets the simulation into "baked" state, and you're safe after that point.

The fun fact is, once a smoke sim is in "baked" state, you can do quite some manipulations without problems. E.g. you can exchange the cache files, or you can even switch the location for external cache files to another directory (e.g. with a newer, augmented build of the cache files). And you can jump around arbitrarily and inspect the state at any frame (I wouldn't call that "playback", since it is quite slow, once the smoke as a reasonable resolution).

And to state the obvious: you need a backup of your precious cache files... I know this is a cost factor, but you just need to price that in. Allocating storage is way cheaper than allocating calculation and work time for re-creating lost cache files.