Make tiles work with additive refinement

[liberostelios]

That's a hack towards an actual solution for #75.

[gkjohnson]

Hi @liberostelios! Thanks for the contribution.

Can you explain a bit more about how this is working. As it is it looks like it will break the "REPLACE" refine mode. What datasets have you tested this on?

[liberostelios]

Hi @liberostelios! Thanks for the contribution.

Can you explain a bit more about how this is working. As it is it looks like it will break the "REPLACE" refine mode. What datasets have you tested this on?

Hi @gkjohnson. First of all, thanks for the response and this library. It's a pretty cool project! Also, my apologies for not offering an explanation in the first place.

This is a small hack I used when I was initially playing with the library. Haven't completely figured out the architecture, but when I came across this check I realised that I could force the traversal to continue through a tile's children in case of additive refinement by adding an exception to the original "exit" rule.

I am not sure if this is a complete and reliable solution, but it kind of worked with a dataset I was playing at the moment. It shouldn't break the "REPLACE" refinement in any case, though, as when tile.refine is not "ADD", then the boolean expression is identical to the original one. Please, correct me if I am wrong.

We are working with a dataset of 3D buildings in the Netherlands. Originally I was playing with a small dataset of around 30 tiles which were created with the additive refinement by this. Now working with a much bigger dataset of about 1300 tiles, which is pretty flat though (just a root with all the tiles as immediate children) and uses the "REPLACE" refinement.

We are planning to use this library as part of our viewer for the 3D BAG project. The plan is to server all building of the Netherlands through our viewer.

[gkjohnson]

Sounds great. Thank you for sharing your project and data -- it's always gratifying to see where our projects get used so I'll look forward to seeing your project evolve!

I am not sure if this is a complete and reliable solution, but it kind of worked with a dataset I was playing at the moment.

This is definitely a feature I'd be interested in getting added to the renderer so if you'd like to help with that it would be greatly appreciated. I'd like to make sure we're confident in adding it, though.

Looking at the spec for refinement it looks like there are a couple things to consider.

• ADD and REPLACE can be used in any order relative to each other like so:

REPLACE
└ ADD
  └ REPLACE
    └ ADD

• It looks like the refine field is optional and should inherit the parents setting by default, which isn't currently being done.

Not that those necessarily have to be done together but just to define the full scope of the behavior a bit more completely.

It shouldn't break the "REPLACE" refinement in any case, though, as when tile.refine is not "ADD", then the boolean expression is identical to the original one. Please, correct me if I am wrong.

Looking at it a bit more closely it's not as broken as I initially thought but I believe it does change the behavior quite a bit because the return statement has been removed. Previously if children weren't ready to be rendered or weren't rendered the previous frame then the parent tile would be loaded / rendered and then the immediate children would be downloaded. Now without the return statement removed children are always traversed and always loaded even if the parent tile hasn't finished loading yet. Because of the childrenWereVisible condition it won't consistently display duplicate tiles but there will be one frame where a parent tile and child tile are displayed at the same time and the parent tile will no longer wait until all child tiles have finished loading to render the children which can cause visible gaps in the terrain.

The traversal algorithm is pretty delicate and a bit complicated so it's not really straight forward to trace what will happen. In particular it gets complicated when you consider the behavior of the traversal as the visible set of tiles is only partially loaded. If you have any suggestions on how to make it more approachable and easier to understand I'd be happy to hear them!

At the moment I'm thinking the best approach would be to track whether all children are of "ADD" refinement in this block of code to determine whether or not the parent tile should render. It would reset every time a used "REPLACE" tile is found to account for the previously mentioned spec behavior. Do you have an example dataset that uses "REPLACE" for testing? It would be great to have a more realistic dataset on hand to test the behavior.

Thank you again!

[liberostelios]

Thanks for the explanation. I have missed the importance of the removal of return here, tbh.

I really need to get familiar with the way things are handled by the renderer and the traversal algorithm, to be honest. Currently I am hacking things around to make things work and chances are I am breaking stuff 😬 For instance, our dataset has an empty root tile which, iirc, previously would immediately stop the traversal. I slightly changed the code to do that. To be honest, I mainly opened this PR with the intention to open the discussion about this (and other future) potential contributions here.

We also have a slightly "peculiar" dataset, at this point: to us, tiles of the same level should still be treated independent of each other according to their SSE which, I think, is not the case here. Currently, as soon as the traversal enter level 1, it will aggressively try to load all leafs. I guess we are out of the specs, but for us it would be more fit that tiles are omitted based on their SSE, same as they are if they are out of the frustrum. I guess we'll tackle that by building a deeper tileset (either one with many empty parents or one that uses additive refinement and progressively introduces buildings) but this is part of the experimentation we are undertaking.

I'll study your ideas with respect to the code and get back to you with more concrete remarks.

[gkjohnson]

Please do let me know if I can help clear anything up regarding how the traversal functions. Like it I said it's a bit complicated so it's probably inevitable that you break things first :D -- I definitely appreciate help and feedback!

We also have a slightly "peculiar" dataset, at this point: to us, tiles of the same level should still be treated independent of each other according to their SSE which, I think, is not the case here.

This is due to the loadSiblings option, which ensures that sibling tiles are loaded at the coarsest level to every currently visible tile that is loaded, as well. This is so we can perform raycasting against tiles that are offscreen and (eventually, not sure how to do this at the moment in three.js) render shadows from non visible offscreen tiles. "Visible Tiles" are those that are visible in the camera frustum while "Active Tiles" are those that are considered to exist for the sake of raycasting / shadows. You can see the behavior by playing with those settings in the demo page and looking through the third person view. This behavior and intent should probably be clarified in the docs and it might even make the most sense to default this to false.

I guess we are out of the specs, but for us it would be more fit that tiles are omitted based on their SSE, same as they are if they are out of the frustrum.

If loadSiblings is set to false then it will only load tiles that are visible within the frustum. However regarding traversal and SSE if a parent tile does not meet the SSE requirements then it will start loading and rendering all the child tiles with the assumption that all the child tiles are needed to fill the space occupied by the now unrendered parent. For additive refinement I think the type of behavior you're describing is what I'd expect.

Generally if you're just representing your data as a mostly flat list of tiles (like the one you linked above) you're going to lose a lot of the benefit to this format especially when zooming out. Our datasets typically only use 2-4 children per tile so the tree can be culled and traversed quickly.

[liberostelios]

Please do let me know if I can help clear anything up regarding how the traversal functions. Like it I said it's a bit complicated so it's probably inevitable that you break things first :D -- I definitely appreciate help and feedback!

Thanks a lot! I will raise any questions.

This is due to the loadSiblings option, which ensures that sibling tiles are loaded at the coarsest level to every currently visible tile that is loaded, as well. This is so we can perform raycasting against tiles that are offscreen and (eventually, not sure how to do this at the moment in three.js) render shadows from non visible offscreen tiles.

The demo was crucial to understanding many things, although I did find the documentation quite informative already, given it's concise form. I think loadSiblings is nice as an option, but we have disabled it for now.

For shadows, I think you would have to write your own shader for such a thing and I am not sure if it's worth it. I assume having a slightly bigger frustum than the actual one and loading/rendering adjacent non-visible tiles should be enough for the shadows to exist.

If loadSiblings is set to false then it will only load tiles that are visible within the frustum. However regarding traversal and SSE if a parent tile does not meet the SSE requirements then it will start loading and rendering all the child tiles with the assumption that all the child tiles are needed to fill the space occupied by the now unrendered parent. For additive refinement I think the type of behavior you're describing is what I'd expect.

That's exactly the behaviour I was talking about. Our problem now (again, given how peculiar our dataset is) is that when you look towards the horizon the tiler will greedily try to parse as many tiles as it can in order to fill the parent's space. Nevertheless, if there was an option of occlusion based on distance/SSE that would make more sense from our flat tiles, as I think most GIS people would perceive it. A narrower frustum should solve this as well, but then you give away the view of other things (e.g., in our case we want to have a terrain that should be able to fill the horizon).

I understand this sounds as a distorted view of 3D tiles, but is a problem of getting used to a different tiles paradigm based on what we are used to with technologies like WMS-T etc in GIS. So i am just mentioning it here as something that others might struggle to comprehend.

Generally if you're just representing your data as a mostly flat list of tiles (like the one you linked above) you're going to lose a lot of the benefit to this format especially when zooming out. Our datasets typically only use 2-4 children per tile so the tree can be culled and traversed quickly.

I think we could solve this by adding one or two layers between the root and the leaves. As far as I understand, there are two options for us:

• add the two levels as empty tiles and let the leaves have the actual content with "REPLACE" refinement, or
• use additive refinement for the new levels and the leaves.

For the second case, we should implement this PR. But for the first one, I am still not sure if it would work either. I've hacked this at some point to force SSE calculation for empty tiles which should allow the traversal to work for them. But I am still not sure if that's an ideal case.

[gkjohnson]

For shadows, I think you would have to write your own shader for such a thing and I am not sure if it's worth it. I assume having a slightly bigger frustum than the actual one and loading/rendering adjacent non-visible tiles should be enough for the shadows to exist.

The shader is less of an issue I think -- we can replace the shaders for each tile with a native three.js material that supports lighting. The issue is that frustum culling happens independent of the three.js renderer so tiles outside the camera view are not added into the scene and therefore cannot be rendered to the shadow buffer. Now that I think about it it could just be a matter of enabling activeTiles to render and setting frustumCulled to true when for the tiles only when rendering active tiles for shadows, though. I'll make another issue for it.

Regarding the tiles issue -- I'm less familiar with WMS-T and GIS but I expect they would try to fill any gaps with child tiles, too?

add the two levels as empty tiles and let the leaves have the actual content with "REPLACE" refinement, or

If a tile is empty it is skipped during the traversal until children are found with content. The spec mentions that empty child can be used to speed up frustum culling traversal or something like raycasting. It's mentioned in this section. I'd say even for your pretty flat hierarchy it's probably worth grouping bounds into a quadtree-like structure if you don't intend to generate more levels of detail tiles.

use additive refinement for the new levels and the leaves.

I think this is your best bet if I'm understanding things right. The tiles would then only be added once they meet the error threshold.

[gkjohnson]

I just merged #108 which added support for ADD refinement as well as some test data so I'm gonna go ahead and close this.