[Feature Request] Text Overlays

[lmcinnes]

The ability to overlay text layers for labelling / annotation would be very useful. Some examples include in ThisNotThat (see the plot at the bottom of this page) or Atlas from Nomic eg this example. Looking through the regl-scatterplot examples it looks like this is relatively tractable. Some questions remain as to the exact API that would work well.

Javascript is not a language I know well, but I would be willing to try and get a version working and try to make a PR if you feel like this would be a useful addition. I'm also happy to discuss options on what the API/options exposed on the python end should be.

[flekschas]

I think this would be a super useful feature and it should definitely be feasible. I think the two big questions are around the API design and the best way to implement this.

API Design

Regarding the API design, I think a natural/fitting way to approach this could be a new function called text() or label(). I'm thinking of something like:

scatter.label(by="name", color="black", outline="white", size=16, max_labels=50, importance="score")

If the name column is of type str, int, float, etc. then we could assumes that each point is associated to a label and that there's some kind of importance score to determine which labels to show at some zoom level.

And in the case that name is categorical, we could assume that we only want to label each category once. (This should lead to a labeling similar to the deep scatter example from Nomic).

Technical Implementation

This will be interesting. Ideally, the text is rendered with WebGL as well but implementing this from scratch will be tough. CandyGraph did a really good job here. Maybe we can take inspiration. Alternatively, I wonder, if we could use the Canvas API. It's slower than WebGL but offers more flexibility when it comes to rendering text and is easy to work with. And assuming we'll limit the number of labels to 50 or so by default anyway, the performance might be fast enough. The biggest question will be how to combine/compose the WebGL and Canvas rendering.

[lmcinnes]

I was actually looking at Deck.gl for their TextLayer as inspiration, particularly the ability to layer text over top of the scatterplot with potentially independent xy positioning for the the text. Their CollisionFilter extension applied to text is particularly appealing but well beyond my capacity to implement. If I was going to implement anything I think it would be something simple very much cribbed from the text label example in regl-scatterplot, assuming I am interpreting how that works correctly. That wouldn't be WebGL, but should suffice as long as you don't want to many labels on screen at a time, which should be manageable.

I may try and take a go at this in a few weeks if I get some time. Thanks for the encouragement.

[flekschas]

Wow, the collision filter seems to work really great! The biggest question is how well deck.gl integrates with regl-scatterplot and how much it depends on geospatial data. Just looking at the view state of the basic examples (https://deck.gl/docs/get-started/using-standalone#using-deckglcore) makes it seem like everything is based on long/lat, which is not really how regl-scatterplot works. There might be a way to integrate the two libraries but it might require a bit of a hack to translate between regl-scatterplot's camera view matrix into the long/lat/zoom view state of deck.gl