Tweaking Continous saving of the game

Florian

Active Member
Contributor
Architecture
Currently the game only saves when it terminates properly. If the game crashes you loose all your work of your current game session.

My goal is to change this by making the game save continously, so that you loose only the very recent work when the game does indeed crash. This also helps with memory consumtion, as currently unloaded chunks only get compressed. Once they get saved, their memory can be reused.

This is my current progress:

When a certain amout of time has passed since the last save, or when a certain number of chunks have been unloaded then the game triggers an auto save automatically.

The saving gets currently done in two phases:

  • Phase 1: On the main thread a memory snapshot of the current game gets taken. This ensures that the game state is consistent and that it can not happen that the ongoing game confuses the save process. In order to reduce the memory usage for doing so, the chunk data does not get duplicated. Instead it gets flagged as "needs to be copied before write" until the save process is completly done. Then this virtual flag gets reverted and if there were no changes during the save no extra memory got allocated .
  • Phase 2: In a separte thread, the previously taken snapshot gets written to disk. This is done in a semi atomic way: Even if the saving gets interrupted by a crash or instant shutdown, the save game folder will remain in a state which can be "repaired". The reparing gets automatically done at the next startup.
The current duration of the phase 1 variates a lot. It ranges from not notiable (20ms) to noticable freezes (1s) on other worlds/situations/setups.

If you are curious on how it looks code wise, you can have a look at my autosave branch:

https://github.com/flo/Terasology/commits/autosave

It is theoretically possible to reduce the time on the main thread further, but it's tricky to get right.

Once concept I am thinking about is to have the game fill two lists as entties get modified and deleted:
  • A list of all deleted entities since the last save
  • A list of all added and changed entities since the last save
When it is time for taking a snapshot, this list could be taken. In their place two empty lists get placed.
The change delta gets then applied to a copy of all persistent entties which is private to the saving thread.

Thus no entities need to be copied or encoded in phase 1. All that needs to be done for entities in phase 1 would then be the swapping of two lists which takes as good as no time.

There are a few details to it that makes it a bit tricky, but if the save pauses turn up to be annoying that might be a good way to solve them.
 
Last edited:

Esereja

Active Member
Contributor
good to see this coming. I have been thinking necessity of auto save, but forgot it as often while working on more interesting things.
 

Florian

Active Member
Contributor
Architecture
I updated the post. I am currently thinking about proposing the current version of my autosave branch for a pull request. The main drawback is currently that the save pauses can be annoying. They range from 20ms to 1s depending on the world/setup/situation
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Up to a full second does sound sucky, but being only in limited scenarios and in the absence of any saving without it I'd say move forward with a PR :)

Probably easier to improve it further with the code merged anyway!
 
Top