Page MenuHome
Create Task
Maniphest T45570

Memory leakage in Sequencer
Closed, ArchivedPublic
Actions

Description

System Information
Windows 7, GeForce

Blender Version
2.74a 64 bit, hash: c27589e

Short description of error
Blender memory consumption is not limited during playback in Sequencer or rendering using Sequencer.

Exact steps for others to reproduce the error
Sorry for not applying a .blend file, but I think that it is not hard to reproduce.

I'm doing a video montage in Sequencer using a lot of video strips, some of them, of course, are just different cutted versions of one video file. The whole animation is about an hour long.

When I try to render my file the Blender crashes after consuming all of available memory. So I can render only ~5 minutes of my project at a time.

Also, there's the button in Sequencer named 'Refresh Sequencer', when I press it, sometimes the memory usage drops down to ~200 MB and sometimes Blender immediately crushes.

I have noticed that the most consuming case is when you have many frequently changing video strips.

Out of memory case log:

Calloc returns null: len=8294400 in imb_addrectImBuf, total 1360638192
Error: EXCEPTION_ACCESS_VIOLATION

'Refresh Sequencer' crash log:

WM_operator_last_properties_init: loading previous properties for 'SEQUENCER_OT_refresh_all'
...
   READ: strID=1 (VID: 0) dts=55951360 pts=55951360  KEY
   READ: strID=1 (VID: 0) dts=55952384 pts=55952384  KEY
   READ: strID=1 (VID: 0) dts=55953408 pts=55953408  KEY
 ->READ: strID=0 (VID: 0) dts=38063025 pts=38064026  KEY
... (and many more READs)

Sorry for not debugging it, I'll try to provide any possible information you ask.

Memory disparity image:
http://i.imgur.com/kb7Ff5t.png

MP4 files used:

Stream #0:0(eng), 15, 1/30000: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1920x1080, 1/60000, 20304 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 60k tbc (default)
Stream #0:1(eng), 1, 1/44100: Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 125 kb/s (default)

Related Objects

Event Timeline

Real Name (uvx3) created this task.Jul 27 2015, 4:45 AM
Real Name (uvx3) raised the priority of this task from to 90.
Real Name (uvx3) updated the task description. (Show Details)
Real Name (uvx3) added a project: BF Blender.
Real Name (uvx3) edited a custom field.
Real Name (uvx3) added a subscriber: Real Name (uvx3).
Aaron Carlisle (Blendify) added a project: Video Sequencer.Jul 27 2015, 5:11 PM
Aaron Carlisle (Blendify) added a subscriber: Aaron Carlisle (Blendify).

Also after rendering I have had a problem were memory is not freed afterwards

Sergey Sharybin (sergey) lowered the priority of this task from 90 to 30.Jul 28 2015, 12:24 PM
Sergey Sharybin (sergey) added a subscriber: Sergey Sharybin (sergey).

It is important to attach all files needed to reproduce the issue. Something which sounds obvious to you is not obvious for everyone. We're using sequencer on a daily basis here in the studio for Gooseberry edit and don't have memory leaks detected so far.

As for the crashes -- it'll help if you can isolate it a bit more. Crashes which happens sometimes to you are areally tricky to nail down from our side.

Real Name (uvx3) added a comment.Jul 28 2015, 5:50 PM

I've created a .blend file, which can help detect the error, I hope. It's in attached .7z archive. There's also some gdb output.

Thank you.

mem_leak.7z59 KBDownload

Sergey Sharybin (sergey) changed the task status from Unknown Status to Unknown Status.Jul 28 2015, 6:37 PM
Sergey Sharybin (sergey) claimed this task.

Refresh Sequencer is not really thread-safe button, better not be used during rendering. Committed change which makes sure it's not used during rendering.

As for memory usage -- the memory is being used by FFmpeg context associated with each of the strip. It's possible to make them better memory managed by blender, but for now it's considered a TODO: http://wiki.blender.org/index.php/Dev:2.5/Source/Development/Todo/Editors#Video_Sequencer

So thanks for the report, but archiving it as a TODO now.

Real Name (uvx3) added a comment.Jul 28 2015, 7:05 PM

Sergey, thank you for making Blender :)

Maybe you can advise me some sort of solution to render my project? Now I render it by 3—5 minutes long pieces only. Is there some kind of 'pause render' possibility to safely free the memory during rendering?

Sergey Sharybin (sergey) added a comment.Jul 28 2015, 7:19 PM

You can have a script which modifies render range, invokes animation rendering for a shorter frame range, then clears the cache and continues rendering.

Second idea is to have pre-render hook which will free cache as well, but this wouldn't work in the current Git version because of the thread safety check added recently. Would need to re-consider the fix to keep that usecase working..

Real Name (uvx3) added a comment.Jul 28 2015, 7:28 PM

Ok, thank you.

Philipp Oeser (lichtwerk) mentioned this in T55366: Blender development todo list -- Editors.Jun 7 2018, 2:37 PM