Page MenuHome

Speeding up creating objects
AbandonedPublic

Authored by Erik Abrahamsson (erik85) on Dec 18 2020, 1:03 PM.

Details

Summary

The main reason I looked into this is I'm helping with the BlenderBIM add-on which can import IFC-files. The problem is many buildings can have several thousands of objects (my biggest IFC imported so far had 70k objects) and import is extremely slow because of the duplicate name check.

This patch probably needs some work if the general idea is accepted.
What it does is:

  • Adds a GSet to Main to speed up checking if an ID name is free to use instead of looping through a ListBase. (improved from 9s to 5.8s to add 10,000 objects with random names)
  • Switches to faster versions of atol and isdigit (7% speedup when separating a mesh to 10,000 new objects on Windows)

This script can be used to add 10,000 objects:

import bpy
import random, time

scene = bpy.context.scene

random.seed(12345)

N = 10000
S = 1 / pow(N, 0.333)

vertices = [ (0, 0, 0), (S, 0, 0), (S, S, 0), (0, S, 0) ]
faces = [ (0, 1, 2, 3) ]

l_random = random.random

last = t0 = time.time()
prev_tdiff = 1

for i in range(N):
    
    x = l_random()
    y = l_random()
    z = l_random()
    
    msh = bpy.data.meshes.new('mesh %d' % i)
    msh.from_pydata(vertices, [], faces)
    msh.update()
    
    obj = bpy.data.objects.new('object %d' % i, msh)
    obj.location = (x, y, z)

    scene.collection.objects.link(obj)

    
now = time.time()
print('Create time: %.3fs' % (now - t0))

Diff Detail

Repository
rB Blender

Event Timeline

Erik Abrahamsson (erik85) requested review of this revision.Dec 18 2020, 1:03 PM
Erik Abrahamsson (erik85) created this revision.

As an addon developer helping users to create large and complex scenes, objects creation being slow has been an issue in Blender since... forever. So this improvement is welcome! Improvements below 100% are still far from enough in practice though. Time increases exponentially it seems, so at around a few tens of thousands of objects, it's practically useless to try to add more. This number is low when making large scenes. Best would be a linear (or lower) time increase, not exponential.
Not trying to compare apples to oranges, but in Unity for instance, object creation is much lighter.
Anyway, good to see improvements in that direction :)

Erik Abrahamsson (erik85) edited the summary of this revision. (Show Details)

I had to separate the hash lists for objects and meshes. Now it should behave as expected.

Campbell Barton (campbellbarton) requested changes to this revision.EditedMay 25 2021, 12:50 PM

This is more a design question - if we want to support this or not.

Also if a gset is being added for each ID this raises the question if we should support a GHash to add fast name -> ID look-ups.


Suggested to handle this as a design task before going into patch review any further.

Note, this patch in it's current form is not correct since duplicate names can exist within it the ID data-base names are only unique within a library, tagging as needing changes although again I think is something that should be agreed on before handling this as a regular patch.

source/blender/blenkernel/BKE_main.h
196–197

It's a bit strange only to support this for two ID types, this should be GSet *id_names[INDEX_ID_MAX].

This revision now requires changes to proceed.May 25 2021, 12:50 PM

A bit of benchmarking on my side shows that when creating objects, on my machine, I start by creating 250 objects every 0.2 seconds. By the time I hit 30,000 objects, it slows down to creating 250 objects every 5 seconds (25 times slower).

Having object counts exceeding 50,000 is quite common on projects in large commercial firms.

Let me know if I can help in any way possible - if implemented, this would benefit the entire field of AEC industry users.

So that this patch which can have a large impact on add-on devs doesn't go to waste, this patch seems to be two parts, only one of which seems to be debated. Can this aspect: "Switches to faster versions of atol and isdigit (7% speedup when separating a mesh to 10,000 new objects on Windows)" which results in a 7% speed up be merged?

Ping @Erik Abrahamsson (erik85) :)

Also, is there any current workaround the add-on devs can do to prevent the slowdown due to the duplicate name check but still have control over names?

So that this patch which can have a large impact on add-on devs doesn't go to waste, this patch seems to be two parts, only one of which seems to be debated. Can this aspect: "Switches to faster versions of atol and isdigit (7% speedup when separating a mesh to 10,000 new objects on Windows)" which results in a 7% speed up be merged?

Ping @Erik Abrahamsson (erik85) :)

Also, is there any current workaround the add-on devs can do to prevent the slowdown due to the duplicate name check but still have control over names?

Hey Dion, this would need some work and now there's D14162 that is similar.. I'll wait and see if that helps.
Regarding the atoi thing, I'll have to try that again and if it's still faster or if I did something wrong there. It depends a lot on compiler settings I think.