Page MenuHome

Auto-save improvements.
Needs RevisionPublic

Authored by Jacques Lucke (JacquesLucke) on Apr 14 2022, 10:41 AM.
Tokens
"Love" token, awarded by wilBr."Like" token, awarded by campbellbarton."Love" token, awarded by Tetone."Like" token, awarded by hadrien."Love" token, awarded by HEYPictures."The World Burns" token, awarded by EAW."Love" token, awarded by billreynish."Like" token, awarded by CarbonSmasher."Love" token, awarded by manitwo."Love" token, awarded by Leul.

Details

Summary

While auto-saving works to some degree, I find it much less useful than it could be when I want to depend on it myself. This patch implements a couple of improvements.

  • Change save interval unit from minutes to seconds. For my own work, I'd like to set the interval very low, because Blender crashes a lot during development and basic testing.
  • Don't auto-save when the current state of the file is already saved (either as auto-save or normal save).
  • New "Recover Last Auto Save" operator that finds the most recent auto-saved file and opens it. That's much more convenient when recovering after a crash compared to picking the right file in the /tmp/ directory.

On top of that, to unify things a bit, it seems reasonable to get rid of quit.blend and use the auto-save system for that as well. quit.blend is just "auto saving" as well really.
So instead of writing quit.blend when Blender quits, we just call wm_autosave_write instead. That might also help protect the user when quitting multiple instances of Blender, since auto-saving encodes the process id in the file name.
The "Recover Last Session" operator on the splash screen should then just do what the new "Recover Last Auto Save" operator is doing in this patch (find the latest auto-save).

Does that sound like a reasonable change?

Diff Detail

Repository
rB Blender
Branch
auto-save-improvements (branched from master)
Build Status
Buildable 21631
Build 21631: arc lint + arc unit

Event Timeline

Jacques Lucke (JacquesLucke) requested review of this revision.Apr 14 2022, 10:41 AM
Jacques Lucke (JacquesLucke) created this revision.
Jacques Lucke (JacquesLucke) retitled this revision from Auto-save improvements. to Auto-save improvements. (WIP).Apr 14 2022, 10:50 AM
Jacques Lucke (JacquesLucke) edited the summary of this revision. (Show Details)
Jacques Lucke (JacquesLucke) edited the summary of this revision. (Show Details)

This solves an issue with multiple Blender instances open, where - because Blender autosaves whether there are changes or not - the instance that saved the latest autosave file at any given time is essentially random.

eg. we have Blender instances A and B:
A autosaves, A crashes, B autosaves, user opens new Blender instance and reloads most recent autosave expecting A's, gets B's instead.

With this patch, the likelihood of encountering this is now minimal (unless the user sets a very large autosave interval).

Jacques Lucke (JacquesLucke) retitled this revision from Auto-save improvements. (WIP) to Auto-save improvements..May 4 2022, 2:51 PM

This change would be useful to me too!

I assumed the recover last session option wasn't working in the past, I expected it to use auto saves when it really just uses quit.blend. So unifying the two sounds great.

I wonder if there's a need for the Auto Save... item in the "Recover" submenu anymore. What purpose does that serve now? If not, and quit.blend is unified too, maybe the whole submenu could just become a single item.

release/scripts/startup/bl_ui/space_userpref.py
1478

I guess this is out of scope of this patch, feel free to ignore it.

This property would probably make more sense if it was called "Interval" rather than timer. I also wonder why the units aren't displayed in the button. Maybe that was just never implemented.

source/blender/makesdna/DNA_windowmanager_types.h
143

Might as well make this a char?

source/blender/windowmanager/intern/wm_files.c
156

You used true for this elsewhere, maybe use false here?

Generally I think these improvements are nice.

  • Not sure about changing the unit to seconds. As dev you may have an untypical workflow. But how useful is that for "normal" users? Do we really need to make them calculate seconds unnecessarily? :)
  • Replacing quit.blend with the auto-save makes sense to me. But I do see an issue: Restore Last Session may also be used in a session that has been going on for long enough for a new auto-save to be stored. It should still bring you back to the previous session then, not to the latest auto-save of the current session. For example when opening a new file (in a new session) just to play around with something for some minutes, trusting Blender to still let you go back to the previous session when done playing. It seems like the previous session is lost then (it isn't, because the auto-save is still present, but you can't expect users to know that). A possible solution would be to still prefix the file name with quit- which Restore Last Session uses but Restore Last Auto Save treats as normal auto-save.

Also worth noting that this still doesn't solve the IMO biggest issue with auto-save & quit.blend, which is that on Linux they are stored in /tmp/ which is cleared on every reboot(!). Separate issue, but a more important one to address still.

Not sure about changing the unit to seconds. As dev you may have an untypical workflow. But how useful is that for "normal" users? Do we really need to make them calculate seconds unnecessarily?

Why should my workflow not be supported here. Could also keep using minutes and use floats instead. Either way, think this is a setting that most users don't touch anyway, and if they do, they probably set it once and then forget about it, so I don't care too much about them having to compute seconds.

It should still bring you back to the previous session then, not to the latest auto-save of the current session.

I think this is just a very different feature. Being able to go back to a file that I edited before might be nice, but it's not really something I expect Blender to handle for me (if I plan to go back, I can just save the file).
The use case I mainly care about right now is that I don't want to loose work when Blender crashes or I accidentally close it. When that happens I try to get back the data immediately and don't work on another file for a while.

Also worth noting that this still doesn't solve the IMO biggest issue with auto-save & quit.blend, which is that on Linux they are stored in /tmp/ which is cleared on every reboot(!).

That's certainly an issue, but definitely not the biggest issue for me. That's really only an issue when the entire computer crashed AND when auto-save would have helped in the first place.

Campbell Barton (campbellbarton) requested changes to this revision.May 5 2022, 8:45 AM

Generally LGTM, notes inline.

source/blender/windowmanager/intern/wm_files.c
1881

This could be called directory even though this called path currently.

1929

I'd rather avoid adding calls to BLI_make_file_string (this function does quite a few things make it difficult to know what the caller intended to happen).

This change uses it on all platforms, not just WIN32. I'd rather this stay WIN32 only.

2007–2016

Prefer not to early return in this case as it adds duplicate function call for no real gain.

2008

An argument could be made only to check wm->file_auto_saved although I don't have _such_ a strong opinion.

You might for e.g. be saving regularly onto a network drive, loose connection ... then try load the auto-save which could be outdated as each time it checked you happened to have saved. A similar case for saving to a USB drive that ends up corrupting your data, or have a second instance of blender open and happen to overwrite this file with an older version.

I think it'd be better just to check file_auto_saved. Then there is a guarantee that the auto saved file is never older then you have set in the timer - no matter what.

3056

Since users may be hoping to recover lost work - silently failing isn't so nice.

This could report an error Unable to find any files with an "_autosave.blend" extension in "/path/to/temp". So it's obvious this action failed.

This revision now requires changes to proceed.May 5 2022, 8:45 AM

This sounds great. Regarding what Hans said about leaving only the recover last option, I personally think it's worth keeping both options. I don't know how common this is for other users but on occasions I'll have two instances of the same blend opened (say, because I partially regret a change I made but if I undo I loose some work I do want to keep, so I reopen the last saved version in a different instance and copy what I need from there). If blender crashes on those occasions, sometimes the last autosave version is not the one I want. I know it sounds convoluted but it's happened more than once and I thought I'd mention it.

Generally I think these improvements are nice.

  • Not sure about changing the unit to seconds. As dev you may have an untypical workflow. But how useful is that for "normal" users? Do we really need to make them calculate seconds unnecessarily? :)

The internal representation could be changed, keeping the user exposed value in minutes, so 1.0 would remain a minute, 0.1 would be 6 seconds... since < 1 minute is a fairly niche use case (mainly for developers).
Either way I don't think users will change this value often, although if someone wants to set the value for 10 minutes, for e.g. using seconds isn't very user friendly.


NOTE: it'd be good to split design discussion about auto-save out of this diff, as the topics are about other possible changes.
  • Replacing quit.blend with the auto-save makes sense to me. But I do see an issue: Restore Last Session may also be used in a session that has been going on for long enough for a new auto-save to be stored. It should still bring you back to the previous session then, not to the latest auto-save of the current session. For example when opening a new file (in a new session) just to play around with something for some minutes, trusting Blender to still let you go back to the previous session when done playing. It seems like the previous session is lost then (it isn't, because the auto-save is still present, but you can't expect users to know that). A possible solution would be to still prefix the file name with quit- which Restore Last Session uses but Restore Last Auto Save treats as normal auto-save.

As this patch doesn't restore the current session I don't think it's so bad, nevertheless you could always open Blender move a cube and now the previous "Latest Session" gets replaced.

Restore Latest could show a menu of auto-save files, ordered by age, (where you'd typically just pick the first). In that case it's close enough to replacing the existing Restore Autosave operator we could consider replacing the existing operator.

Another issue with removing quit.blend is the temp directory would likely become full of {$PID}_autosave.blend for every time you open Blender and start making edits before saving.

When the saved name is used the number of files is at least limited to files you're saving locally. But when the PID is used, many more files can be created, which could be an issue for users that don't clear their temp directory on reboot.

This could be mitigated by...

  • Using a single name for an unsaved file when exiting (so exiting an unsaved file doesn't leave the file named with it's unique PID), the PID still needs to be kept at runtime to avoid collision.
  • Limiting the number of auto-saved files (on exit perhaps), default to ~10 or so. To prevent creating large numbers of auto-saved files - we might want to do this anyway.

I'd also like to suggest for compression to be an option for Auto Save just as it is for normal saving.

Today I realized that Blenders autosave does not use the compression setting from the normal saving user preferences.
My compressed files I work with are 2-3GB usually. Non compressed these are 15GB in my case.
While I am well aware of the stalls blender has for me when autosaving, I did not occur to me that it may be due to them not being compressed.
Worse yet, I realized my relatively new SSD now already sits at 115TBW after just a few months. Understandable with 15GB being written every 2 or 3 minutes.

@Pablo Vazquez (pablovazquez) and I had a look at this together. Our feedback:

  • These are nice changes mostly.
  • Save interval in seconds is odd, we'd rather not do this. Inputting a float (e.g. 1.5 for 1m30s) would be fine.
  • Re merging auto save and restoring last session [not part of this patch]: This would indeed be nice, but Restore Last Session should always restore the state from the last time Blender was closed. I.e. not restore the last auto save. So we do have

I agree auto save files should be compressed by default.


Why should my workflow not be supported here. Could also keep using minutes and use floats instead. Either way, think this is a setting that most users don't touch anyway, and if they do, they probably set it once and then forget about it, so I don't care too much about them having to compute seconds.

Because we shouldn't build Blender around dev workflows? :) Minutes are simple, clear and sufficiently granular for regular usage. No need to multiply or divide to get the unit you're thinking in anyway. Supporting floats is fine (like mentioned above).

I think this is just a very different feature. Being able to go back to a file that I edited before might be nice, but it's not really something I expect Blender to handle for me (if I plan to go back, I can just save the file).

It's an existing feature that the suggested change would break, unless we consider the mentioned case in the design. The whole quit.blend design is supposed to bring persistent sessions to Blender, which never worked out that for various reasons. It's nevertheless a feature with plenty of potential, and Ton feels strongly about it too. Knowingly breaking or removing it is not an option at this point.

That's certainly an issue, but definitely not the biggest issue for me. That's really only an issue when the entire computer crashed AND when auto-save would have helped in the first place.

Well, plenty of people shut down their computers often (daily or more), for various reasons. Any auto-saving that looses data on reboot will become pretty much useless to them.