Sorry to derail a bit, don't have a twitter, and you're a bit hard to contact...
I see you changed your course towards aligning this patch with grease pencil. In regards to architectural work and all of us craving for a pure-engineering-FreeCAD-like realtime line engine, will this addition still give a possibility to work completely in such an environment?
I was kinda looking forward to finally have a workbench engine to create and edit my objects in pure line fashion, like some of your early presentations.

https://media.blendernation.com/wp-content/uploads/2018/06/yiming-wu-blender-2018-06-07-17-40-25.jpg

In D8758#234396, @Okavango (Okavango) wrote:

I was kinda looking forward to finally have a workbench engine to create and edit my objects in pure line fashion, like some of your early presentations.

Hi! It's within the plan. The main reason for removing the real-time engine is because it's too pre-mature and we decided that we should keep functions simple and more well integrated into blender. I'm probably gonna do the real-time modes in a overlay engine, but it will has to be a separate patch to ensure quality of the code.

Ah, ok, overlay engine - makes total sense. That's great info!
Good luck with this one too!

In D8758#234403, @Okavango (Okavango) wrote:

Ah, ok, overlay engine - makes total sense. That's great info!
Good luck with this one too!

Thank you bro :)

The variable scope problem will be fixed later.

I think this one is less of a problem, just something that's nice to have consistent. I don't want to make it a requirement for this patch at all, though I do think that it results in more readable, maintainable code.

Here are some more topics we talked about in the UI team meeting:

Reference to camera
The modifier's reference to the the active camera is currently implicit, which doesn't really make sense for a modifier.
The camera should be selected with a property in the modifier. Another benefit of doing it this way is that you could more easily make a temporary camera active to preview the line art.
General UI polish
In general tooltips and UI layouts need to be cleaned up a bit. I'm happy to help provide some guidance on that.
Move settings to modifier
It is strange to have the settings for generating strokes in the scene when the strokes are generated by the modifier.
Wherever possible, these settings should move to the modifier. Then the settings would also be adjustable per-modifier.
The settings in the scene properties should would then be used as overrides or simplification settings where necessary.
Depsgraph integration
Currently there is some flickering when you change settings in the modifier. The guess was because the modifier generation is not properly tied to the dependency graph.
The stroke generation should be integrated with the depsgraph so that the new strokes just replace the old strokes.
This would also remove the lineart toggle currently in the scene preferences, but the modifier viewport visibility option should allow the same use-case of only showing the strokes in render view.
Deleting one object from the target collection does not recalculate the line art. Although the boolean modifier currently has the same issue, this should not be forgotten.
Transparency masks
We couldn't quite figure out what these are supposed to do. I will test some more, but anyway, the general consensus was that we should not be adding flags like the old layer system to new features.
Again, without knowing what this is supposed to do it's hard to offer an alternative solution, but there should be a more user-friendly way to handle this.
Selectability
The options in the add menu should not disable the selectability for the new grease pencil object. This is just too confusing, and it's important to be able to select the object to change its modifier settings.
The new object should also be selected when it's added.
Add menu
The addition of the two options here is helpful. Generally we use "Quick Effects" for this, but there was consensus that the add menu is a better place for these in general.
Miscellaneous

• It can be confusing that the strokes are generated from the perspective of the camera as you navigate in the viewport, especially confusing in render preview mode. However, in the context of this being a modifier, there isn't really any fix for this.
• There should be a quick way to generate Line Art for the entire scene (with the Scene Collection selected). A simple way to do this would be to avoid disabling the "Collection Line Art" item in the add menu when the scene collection is selected.
• The new modifier needs an icon. The UI team can coordinate this.

In D8758#234535, @Hans Goudey (HooglyBoogly) wrote:

Reference to camera
The modifier's reference to the the active camera is currently implicit, which doesn't really make sense for a modifier.
The camera should be selected with a property in the modifier. Another benefit of doing it this way is that you could more easily make a temporary camera active to preview the line art.

line art is calculated in one go for everything in the scene, and the modifier is only used to select specific part of the result for different style choices, so setting per-modifier camera doesn't really work in this case.

If we want to clarify this, we can move line art *modifier* into a new tab, like particle or physics stuff, and from there all the adjustments including the camera can be shown clearly. But how to handle multiple line art modifiers in this form should be further discussed.

General UI polish
In general tooltips and UI layouts need to be cleaned up a bit. I'm happy to help provide some guidance on that.

Ahh that would be very helpful! Please do :D

Move settings to modifier
Wherever possible, these settings should move to the modifier. Then the settings would also be adjustable per-modifier.
The settings in the scene properties should would then be used as overrides or simplification settings where necessary.

As stated above, line art core doesn't do per-modifier calculation, the global settings panel is to tell line art what to calculate, and the modifier is used to select different portion of the result (hence the source selector and line types/occlusion levels etc.)

If we want to simplify this panel, line art can technically scan all the modifiers prior to calculation and decide what to do, but you may still want the global panel as override. (But in current UI setup you can already kind of think it that way)

Depsgraph integration
The stroke generation should be integrated with the depsgraph so that the new strokes just replace the old strokes.
This would also remove the lineart toggle currently in the scene preferences, but the modifier viewport visibility option should allow the same use-case of only showing the strokes in render view.
Deleting one object from the target collection does not recalculate the line art. Although the boolean modifier currently has the same issue, this should not be forgotten.

This has to be done at some point. Currently I'm triggering calculation in generic scene update function, the flicker is caused by first scene update (it ask for data and it's not ready yet, but calculation started immediately), and second update when calculation is done, that's why.

I'm not familiar with depsgraph, I could use some help in using it, do I need to build "nodes" or something like that?

Transparency masks
We couldn't quite figure out what these are supposed to do. I will test some more, but anyway, the general consensus was that we should not be adding flags like the old layer system to new features.
Again, without knowing what this is supposed to do it's hard to offer an alternative solution, but there should be a more user-friendly way to handle this.

This needs some explanation, I'll describe it later in the thread.

Selectability
The options in the add menu should not disable the selectability for the new grease pencil object. This is just too confusing, and it's important to be able to select the object to change its modifier settings.
The new object should also be selected when it's added.

I found out that if you have selectable gp line art, it always gets in the way of model manipulation... Yeah maybe by default we should allow select.

Miscellaneous

• There should be a quick way to generate Line Art for the entire scene (with the Scene Collection selected).

Actually this is doable... I'll add an option in the modifier that allows this to happen.

If this is a modifier/depsgraph feature vs. a rendering feature is a really important design decision that needs to be clear.

Is there some reasoning behind why it was exposed as a modifier in this patch? Was it just an unusual UI decision that we need to change (the way it's now is definitely too weird)? Or is there also an idea that you can apply further modifiers to the result (which I imagine would be very useful)?

I can see practical reasons why it would be rendering feature, particularly for interactivity. And it's still possible to expose such functionality as a modifier later if needed. But can the grease pencil module team (@Antonio Vazquez (antoniov) @Matias Mendiola (mendio)) verify that it is indeed not intended to be an actual modifier now?

In D8758#234554, @YimingWu (NicksBest) wrote:

As stated above, line art core doesn't do per-modifier calculation, the global settings panel is to tell line art what to calculate, and the modifier is used to select different portion of the result (hence the source selector and line types/occlusion levels etc.)

If we want to simplify this panel, line art can technically scan all the modifiers prior to calculation and decide what to do, but you may still want the global panel as override. (But in current UI setup you can already kind of think it that way)

In general the convention in Blender is that artistic look for rendering should be per object or material when possible, and that scene level settings should be mainly for quality or disabling features as a whole for debugging or performance.

There are too many artistic settings that are global now.

In D8758#234612, @Brecht Van Lommel (brecht) wrote:

If this is a modifier/depsgraph feature vs. a rendering feature is a really important design decision that needs to be clear.

Is there some reasoning behind why it was exposed as a modifier in this patch? Was it just an unusual UI decision that we need to change (the way it's now is definitely too weird)? Or is there also an idea that you can apply further modifiers to the result (which I imagine would be very useful)?

I'm more leaning towards the modifier/depsgraph feature. As using modifier is very useful for further adjustments, not like freestyle, all settings are for final image.

In general the convention in Blender is that artistic look for rendering should be per object or material when possible, and that scene level settings should be mainly for quality or disabling features as a whole for debugging or performance.

There are too many artistic settings that are global now.

If you think of it, not really much "artistic" things are global, those arguments are really for controlling the calculation, all the display-related stuff are controlled in the modifier. In workbench, you could have a bunch of user-tunable globals, or like freestyle, there are even more "advanced options" in the viewlayer tab as well. I don't think this is really a big problem as how we think of it, but I'll need some help on depsgraph.

I think there will be some structural tweaks on the evaluation side that could finally allow the smooth updating of the result. Where should I look at? a lot of things seems to be designed specifically for the existing rendering data relationships, where line art->gp path isn't really that much of a "rendering" thing as the gp result is after all a scene component. I'm not sure how should I add such evaluation, If this is doable, then where should I start?

@Hans Goudey (HooglyBoogly) So here I'll explain how transparency works for LineArt:

1. You need a "glass" material in order to see through. This is configured in mesh material Line Art panel, enabling transparency, and enable some bits in the 8 available ones (e.g. enable bit 1).
2. Then in GPencil Line Art modifier, enable transparency, and choose corresponding bits, in this example would be bit 1, and of course you need to set appropriate occlusion levels as well, because you are behind certain layers of faces now.
3. If you have multiple "glass" material set up, using the combination of bits could allow you to do overlapping, Just like in the example set up below.
   
   
   

Option for showing whole scene's result is added. Accessible through Add->Scene Line Art as well as directly in the modifier.

So here I'll explain how transparency works for LineArt:

For me it's a bit hard to understand the use-case for such a complicated method for this. Maybe I'm being naive, but it seems that by examining the use case for those bit masks we could devise a more intuitive solution to the problem.

I'm more leaning towards the modifier/depsgraph feature. As using modifier is very useful for further adjustments,

I agree, this is where the integration with grease pencil could actually be quite helpful. But in order for this to work well, the calculations should actually be called from the modifier stack.
In my testing I've found that I have had to use the scene settings to make important adjustments to get the lines to render properly. In other words, I found the settings currently exposed in the scene essential for the artistic function of Line Art.
For example, if I had two different type of objects that I wanted to use for line art and wanted to adjust these settings individually, the only way to do that would be to make the settings per-modifier.
This is the clear path forward in my opinion-- further integrating with the depsgraph and the modifier stack and using those changes to consolidate as many properties as possible in the modifier.

As stated above, line art core doesn't do per-modifier calculation

Basically my point is that this should change. Since the hard work of actually making these calculations work is already done, I think this is an realistic change.

In D8758#234821, @Hans Goudey (HooglyBoogly) wrote:

Basically my point is that this should change. Since the hard work of actually making these calculations work is already done, I think this is an realistic change.

If we don't load the whole scene, how could I determine the occlusion? Or if we load the subset, then all the occlusion is happening inside there, and it will not care if anything else's interaction with selected ones, so for example, you have a cube partially behind a wall, but you add it to Object Line Art, it will appear as if it's not hidden. Does this sound good to you 🤔 ?

In D8758#234821, @Hans Goudey (HooglyBoogly) wrote:

the only way to do that would be to make the settings per-modifier.

Yeah, so what we have now is that our scene is computed once, and you can choose to style different parts of the scene without doing multiple calculations. Using several modifiers should do the trick.

In D8758#234821, @Hans Goudey (HooglyBoogly) wrote:

For me it's a bit hard to understand the use-case for such a complicated method for this.

You can use such function in scenes with transparent mesh materials. e.g. to render structure lines in a car or a house's window, that way you don't need to hide the transparent mesh and calculate line art again, those will be handled correctly in one go.

In D8758#234776, @YimingWu (NicksBest) wrote:

I'm more leaning towards the modifier/depsgraph feature. As using modifier is very useful for further adjustments, not like freestyle, all settings are for final image.

Be aware that you can't do incremental updates then, the depsgraph is fully evaluated before the viewport is drawn. It will only be evaluated again when changing frames or user edits.

This would mean you implement this feature as a regular grease pencil modifier, generating strokes. Depsgraph integration shouldn't be very difficult, probably it's just a matter of implementing a modifier in a way that is similar to existing modifiers and this might require no direct changes to the depsgraph.

If you think of it, not really much "artistic" things are global, those arguments are really for controlling the calculation, all the display-related stuff are controlled in the modifier. In workbench, you could have a bunch of user-tunable globals, or like freestyle, there are even more "advanced options" in the viewlayer tab as well. I don't think this is really a big problem as how we think of it, but I'll need some help on depsgraph.

The thresholds and angle splitting are artistic choices as far as I can tell. The line types I guess could be useful as global overrides since they are also in the modifier, though I'm not sure how useful it is in practice.

I think there will be some structural tweaks on the evaluation side that could finally allow the smooth updating of the result. Where should I look at? a lot of things seems to be designed specifically for the existing rendering data relationships, where line art->gp path isn't really that much of a "rendering" thing as the gp result is after all a scene component. I'm not sure how should I add such evaluation, If this is doable, then where should I start?

It's not clear to me what you want to do to allow smooth updating.

To reduce any confusion about the modifier part of things I'll explain the way we came here a bit.

At the start LANPR was more of a render engine. IE it would render the whole scene and everything would could be viewed and exported as a big bundle of strokes.

However to make it easier to separate out object and characters, we made it a modifier so you can split out the stroke to multiple GP objects and then add effects and modifiers to the strokes on a per object basis.
This also makes you able to easily bake and edit specific strokes.

So for example, you could bake the background/environment and then only use the environment object for occlusion when working on the rest of the scene.

This makes the workflow more modular and less monolithic.

Now for the start I also wanted everything to be configurable from the modifier (and I still do). We talked about this (GP team, @YimingWu (NicksBest) and I) and the reason why it is currently this way is because of speed and memory usage.

If every modifier evaluates it's own LineArt strokes it will be much slower and take up much more memory.
Especially on the memory side of things, this can easily lead to out of memory situations.
Even if @NickBest optimizes the memory usage even more, on complex scenes this will be a big issue.

So what I suggested if that for certain effects or situations, you can specify (checkbox or so) that you want the modifier to not use the global evaluation method and instead do everything itself.
However this wasn't implemented yet.

So what is happening currently is that the modifiers takes their strokes from the same scene evaluation.
The settings that are global are there because we can't have these differ on a modifier per modifier basis if we want to only evaluate the scene once.
(As the modifier essentially only copy over the strokes that belong to them)

I think we should have both as doing this as a regular modifier (each modifier do everything itself) would be really slow an memory consuming.
In most cases the global settings does not need to be tweaked per object so this is a very nice optimization.

But as I said I 100% think that the option to have it evaluated as a regular modifier (so the modifier does everything itself) should be there because there will be situations were you would want this.

This could simply be done by having the global setting available in the modifier but they greyed out and we have a "use global settings" checkbox that is on per default.

Or hmm, we could also have some kind of "shared settings" system.
So in theory we could have the modifiers specify which evaluation they want to pick strokes from. So you could have scene eval A, B, C. And then pick which one to use from the modifier.
This was the user can optimize for themselves. But I'm unsure how this would be setup and work in a user friendly way.

In D8758#234869, @Brecht Van Lommel (brecht) wrote:

This would mean you implement this feature as a regular grease pencil modifier, generating strokes. Depsgraph integration shouldn't be very difficult, probably it's just a matter of implementing a modifier in a way that is similar to existing modifiers and this might require no direct changes to the depsgraph.

The thresholds and angle splitting are artistic choices as far as I can tell. The line types I guess could be useful as global overrides since they are also in the modifier, though I'm not sure how useful it is in practice.

I think I explained above why it is like this currently. Let me know if things are still unclear.

It's not clear to me what you want to do to allow smooth updating.

It's basically that the UI doesn't lock up when it is evaluating the strokes.
So now we have more of a job system so the modifier wait for the strokes without locking up blender.

Thanks for the info. These kinds of design decisions and trade-offs really have to written down before code review. Please put that information in T80194, and make sure you all agree on it. And then the UI team and I can give feedback and review. Or ask for help on design aspects you are unsure about.

The intent as I understand it is to get a basic version of Line Art ready for master, as a base to build further improvemnts on. We could support both as a modifier or rendering feature in the future. But for the basic version please pick one and then we can ensure the UI and implementation for that works well.

In D8758#234971, @Brecht Van Lommel (brecht) wrote:

The intent as I understand it is to get a basic version of Line Art ready for master, as a base to build further improvemnts on. We could support both as a modifier or rendering feature in the future. But for the basic version please pick one and then we can ensure the UI and implementation for that works well.

The render engine feature has been ripped out. This is purely a modifier feature now as you can't get any strokes without the modifier.
Every stoke shown is now stored in a GP object.
But it is unconventional in the way that we evaluate the scene globally and not on a per modifier basis for speed and memory usage.

So unless there is any NAKs, then most of the things left a basically polishing the presentation and location of settings.

I continued this design discussion in T80194.

I don't need to be a different reviewer. I'm already reviewer as a member of the Grease Pencil team

ED is for editors. the prefix should be MOD.