Uploading groups of shader attributes in DX12 (Buffer and view questions)

[RobertMtx]

I accidentally posted this question in the samples section. Please remove that other post if possible. Thanks!

What would be the most efficient method to upload dynamic shader properties for 30-60 game elements while using the DX12 backend? I've been trying to plan this out for a while, but don't have enough experience with newer APIs yet to know what would work best. I may support Vulkan in the future, but currently dealing exclusively with DX12.

I would have asked these questions in a general forum, but I"m noticing Diligent Engine requires some variations on techniques, compared to the base DX12 API. For example, the way that dynamic buffers are fully uploaded when they are unmapped. I started off mapping small sections of a dynamic buffer and drawing just those small sections, but then realized my entire buffer would be copied to the GPU every time it was unmapped. Anyway, I would really appreciate some advice on some of these details.

In my game engine, concerning just the properties that will be used in shaders, game entities would have two types of attributes: those that rarely change, and those that change every frame. Some examples would be world transform, colors, ambience factors, roughness factors, etc. If each of my rendered elements has a set of all of these types of properties, what would be the most efficient method to get them all to the shader?

I'm hoping to store all of the data into as few buffers as possible, and upload everything at the beginning of each frame, if that makes the most sense. And possibly to separate the less frequently updated attributes into another buffer that is only uploaded when those specific things change.

1. Should I split attributes up based on how frequently they are updated (two buffers/views per shader/pipeline/etc)?

2. Should I use separate individual buffers for each larger shader concept (particles vs skinning vs morph vs ...)? I'm assuming this would require linking/uploading all of the individual buffers at the start of the frame. But this would allow each technique to have specific attributes in the dynamic vs constant buffers.

3. Should I use separate buffers for each individual pipeline state (num_lights=3 vs num_lights=6)? Again, I think this would increase the number of upload calls, but maybe its necessary?

As far as dynamic buffers go, I've looked at most of the tutorials and samples, but still not clear on the following..

4. Would it be better to create a single large dynamic buffer that is used by everything in the engine, which is always mapped, and simply iterate over it until you reach the end, then reset? I used to draw my screen sprites this way with Dx9, and it worked pretty well.

5. And if so, how does one create a shader association (or view) to sections of such a buffer at render time? When drawing primitives from a dynamic buffer with DrawIndexed(), you can specify the start and size of what is being rendered. But when creating shader resource associations, such as to an array of bone transforms, I haven't figured out how to specify the start and size at render-time (after the view is created). If my bone transforms happen to get written to the middle of a large dynamic buffer on frame 26, how do I link that exact location to the shader variable (BoneTransforms array, or etc) each frame?

6. If using a large dynamic buffer with arbitrary runtime offsets isn't the best idea, should I create a separate dynamic buffer per group of frame-based attributes? Or should/can I create a single dynamic buffer that is mapped out at object-setup time, and then split into multiple shader views, with each view pointing to a different section? Does it matter if I use 1 dynamic buffer vs 6?

And last questions..

7. For attributes that are needed in the shader, but do not change every frame, is it possible to upload specific values that exist in such a buffer, without uploading everything else in that buffer that did not change? If the GPU upload is triggered by a value change, it seems wasteful to re-upload the entire buffer of data, but I haven't found anything in the samples yet that demonstrates how to change a subset of an array of data.

8. If uploading subsets of data into a GPU buffer is not possible, does that mean it would make more sense to split such data up into several smaller buffers, to avoid re-uploading identical data as much as possible? Is there some specific size buffer of such data that I should avoid growing beyond because of this?

Sorry for so many questions! Really appreciate any advice! Feel free to answer as many or as few as you want.

[TheMostDiligent]

Hi there! The answers to your question depend on how much data you need to upload to buffer (few bytes vs few KBs vs MBs) and how many updates you need to do (100s vs 1000s vs millions). If you do 100s of updates, then any method will work really. If you do 1000s of updates, then you may need to think more carefully. 100s of thousands of updates definitely require some careful implementation.

1. Should I split attributes up based on how frequently they are updated (two buffers/views per shader/pipeline/etc)?

Generally, yes. If you have some data that does not change, it is better to separate it. On the other hand, if it is only few bytes, you may also upload it to dynamic buffer every time

2. Should I use separate individual buffers for each larger shader concept

I don't know if you mean vertex buffers or uniform buffers. For vertex buffers it is better to only have what you need in the buffer for your model. However, if you have different varitations that require creating multiple copies of the same data, then it is better to have everythin in the same buffer.

3. Should I use separate buffers for each individual pipeline state

No, this is not necessary.

4. Would it be better to create a single large dynamic buffer that is used by everything in the engine, which is always mapped, and simply iterate over it until you reach the end, then reset?

Maybe - hard to tell what exactly will work best for your application. You can take a look at tutorial 10 that uses similar strategy.

5. And if so, how does one create a shader association (or view) to sections of such a buffer at render time?

You will need IShaderResourceVariable::SetBufferOffset

If using a large dynamic buffer with arbitrary runtime offsets isn't the best idea, should I create a separate dynamic buffer per group of frame-based attributes?

It may be a good idea. If you can update your buffer only once and then use offset to access its regions - that should work OK.
But creating small dynamic buffers may also be OK.

Or should/can I create a single dynamic buffer that is mapped out at object-setup time, and then split into multiple shader views, with each view pointing to a different section?

For attributes that are needed in the shader, but do not change every frame, is it possible to upload specific values that exist in such a buffer, without uploading everything else in that buffer that did not change?

Do you mean, update every second element, for instance? You can write a compute shader that does that, but I am not sure this is the best approach. You can update contiguous parts of the buffer with IDeviceContext::UpdateBuffer

This may be helpful.

[RobertMtx]

Thanks a ton for the advice. Really helpful!

I'm still laying out the exact shader attributes that will be used, so I don't have an exact size yet, but I'm guessing it would be less than 256 bytes per element (edit: element = rendered object), with a max element count of around 500. That doesn't include bone transforms for skinning, etc., but standard elements won't actually use 256 bytes, so it should be pretty close. (edit: I did some math and realized bone transforms take a lot more space than I glossed over here, around 2560 bytes per character. So probably need to upload these per draw). Probably at least half of that size would be static data that does not change per frame.

When I asked about updating subsets of data, and if the entire buffer needs to be updated, I was referring mainly to material-like attributes that exist per rendered element, but that don't change very often.

For example, every element in the game may have a color attribute, but very few of the objects will change color every frame, so I could store color into a buffer that perhaps isn't dynamic, along with other similar values that don't change often. So I'm guessing this would be a structured buffer, where each struct in the array contains all of the (mostly static) attributes for all active elements.

At some point, if a single object's color changes (such as through scripting), it will need to be synced from the PC to the GPU buffer. When I do this, will I need to re-copy all of the other data/elements that are stored in that (probably static) GPU buffer, or can I just update this specific single color value? Or possibly an X-sized chunk of it? And if its not possible to update sections of a static-like buffer without re-uploading all of it, does it make sense to split such a buffer up into smaller buffers just to avoid re-uploading the data? Or is the cost so small that it doesn't matter? In my specific case, the buffer would probably be no larger than a typical skinmesh vertex buffer.

I'm assuming re-writing data onto non-dynamic buffers is not cheap, but I probably don't want to store every single attribute into dynamic buffers just in case they change at some point.

When IDeviceContext::UpdateBuffer is used with a usage-default buffer, does Diligent have to re-upload the entire buffer contents to the GPU in the background (DX12)?

Please forgive possible ignorance here. I'm still learning the essentials, and may be confusing some things. Thanks again for your help!

[TheMostDiligent]

It seems to me that you are overengineering at this point. If you need to render 500 different objects with less than 256 bytes of data per object, you can just use dynamic buffers. Mapping dynamic buffers in D3D12 backend is virtually free. I suggest you make it work first and then find the performance bottleneck, if any.

can I just update this specific single color value

You can update continuous parts for the buffer using IDeviceContext::UpdateBuffer. If your vertex elements are stored like this:
Pos, Color, Normal, Pos, Color, Normal ..., you can't copy only the Color part. You can write it from a compute shader. You can also split the data into multiple buffers.

And if its not possible to update sections of a static-like buffer without re-uploading all of it

You can update any region of a USAGE_DEFAULT buffer with IDeviceContext::UpdateBuffer.

I'm assuming re-writing data onto non-dynamic buffers is not cheap

It may not be expensive either, depending on how much data you need to update. Uploading 100s of MBs per frame will definitely be a problem, but uploading 100 Kbs is not a issue at all.

When IDeviceContext::UpdateBuffer is used with a usage-default buffer, does Diligent have to re-upload the entire buffer contents to the GPU in the background (DX12)?

No, only the part of the buffer is updated.

[TheMostDiligent]

The source code of Unmap() doesn't look like it works that way

It does work this way - you can try to run any sample and step through the code.

how the GPU will know which part of the mapped dynamic buffer memory is old vs new

It does not know. Dynamic buffers use coherent memory, so all reads made by the CPU are automatically visible by GPU.

[RobertMtx]

The source code of Unmap() doesn't look like it works that way

It does work this way - you can try to run any sample and step through the code.

I guess I should have said, it doesn't look like it works that way at first inspection. I'm totally willing to take your word for it until I get more familiar with all of this.

how the GPU will know which part of the mapped dynamic buffer memory is old vs new

It does not know. Dynamic buffers use coherent memory, so all reads made by the CPU are automatically visible by GPU.

That's pretty incredible. I did not expect that to be the behavior. Do you happen to know if the memory is lost/unmapped when device resources are taken over, such as if the user tabbed out (assuming these things still happen in DX12)?

Is there any reason to not just store your shader states directly into the mapped memory? I was thinking there would be two different ways to approach it - either you keep a standard CPU block of memory ready to go, then copy that to the dynamic buffer each frame before drawing, or you store the shader states directly into the mapped dynamic buffer memory (updating it one value at a time, on the fly, when it changes). With the second approach, you would have to store all of your active objects (or at least those in the immediate area) in the dynamic memory, even if most of those objects won't be rendered. With the first, you would only need to copy visible objects to the dynamic memory, keeping all others in the CPU block. I guess it would depend on how many objects are in the immediate area, but using the first approach is somewhat hard-coding an active-area entity limit into your engine.

There was one other thing I can't seem to find the answer to. Do you have any idea where dynamic buffer memory is allocated? Is that stored on the GPU or the CPU? I noticed that the GraphicsAdapterInfo::UnifiedMemory value is always set to 0 for DX, and my readout for GraphicsAdapterInfo::SharedSystemMemory is over 4GB, which is larger than my readout for GraphicsAdapterInfo::DedicatedVideoMemory. I'm assuming this means everything except for Dedicated memory is being stored on CPU ram. But still unsure of where dynamic buffers would be placed. The reason being is that if coherent memory is always stored on the CPU, then it would be virtually limitless. I could store nearly my entire game engine into a dynamic buffer. But I'm assuming that is not how it works. Any insights into this? Perhaps I'm confusing two things (coherent memory is not the same as shared memory)?

[TheMostDiligent]

Do you happen to know if the memory is lost/unmapped when device resources are taken over

The system swaps the application memory when it needs.

Is there any reason to not just store your shader states directly into the mapped memory?

This is not how it works. Shaders are internal GPU state handled by the driver along with other state descriptions (blend, depth, rasterizer, etc.).

Do you have any idea where dynamic buffer memory is allocated?

https://asawicki.info/news_1755_untangling_direct3d_12_memory_heap_types_and_pools
https://therealmjp.github.io/posts/gpu-memory-pool/

[RobertMtx]

When the buffer is mapped with MAP_FLAG_NO_OVERWRITE, it contents is not copied. Unless you use UNORDERED_ACCESS flag, dynamic buffers are written directly by CPU and read by GPU and no copies are performed.

I may have stumbled onto a problem, and could use some advice if you have a minute. I noticed that BIND_FLAGS::BIND_SHADER_RESOURCE has a comment noting that it should never be used with MAP_WRITE_NO_OVERWRITE. I believe this is a typo, and it actually means it cannot be used with MAP_FLAGS::MAP_FLAG_NO_OVERWRITE?

But if this is the case, doesn't that mean one cannot map dynamic buffers that contain non-vertex / non-index buffer shader values while supplying MAP_FLAGS::MAP_FLAG_NO_OVERWRITE? The only sample that seems to be using the no-overwrite flag is the data-streaming tutorial, and it uses it with a vertex and index buffer only.

I'm thinking that since I'm shooting for Dx12 primarily, I can probably just leave my dynamic buffers mapped indefinitely, and won't need to worry about the no-overwrite flag. Does that make sense? Does the no-overwrite flag diverge any behavior in Dx12 beyond the mapping/unmapping functions? When you suggested I use the flag, I'm assuming you thought I was mapping vertex buffer data? Or is perhaps the comment outdated and they can now be used together?

Thanks!

[TheMostDiligent]

I noticed that BIND_FLAGS::BIND_SHADER_RESOURCE has a comment noting that it should never be used with

This is only the case for older Direct3D11 runtime. I will remove this comment.

I'm thinking that since I'm shooting for Dx12 primarily, I can probably just leave my dynamic buffers mapped indefinitely, and won't need to worry about the no-overwrite flag.

The contents of any dynamic buffer is discarded at the end of the frame and must be written the first time it is mapped. You can have it mapped, but only within a single frame.

Does the no-overwrite flag diverge any behavior in Dx12 beyond the mapping/unmapping functions? When you suggested I use the flag, I'm assuming you thought I was mapping vertex buffer data?

Any buffer can be mapped, however buffers with BIND_UNORDERED_ACCESS flag will incur a large performance penalty.
This should also be useful for you.