Should `nodesFromRect()` return elements with no area?

Add-ons Development
#1

I have a question regarding nsIDOMWindowUtils#nodesFromRect().

I use this method in VimFx as a way of getting all elements currently inside the viewport. However, I’ve just noticed that the method seems to exclude elements with no area.

An example:

<!doctype html>
<title>test</title>
<p>
  <a href="links.html">Simple link</a>
</p>
<p>
  <a href="links.html">
    <span style="float: left;">Link with floated children.</span>
  </a>
</p>

The above code demonstrates a fairly common scenario on the web. The second link contains only floated children. This means that the height of that link is 0, making its area 0 as well. The link is still clickable, though, because clicks on child elements of links activate the link as well.

For VimFx’s purposes, <a> elements are very important. While nsIDOMWindowUtils#nodesFromRect() does return the floated children, that’s not enough. I need the actual <a> element.

Does anyone know if this is the intended behavior of nsIDOMWindowUtils#nodesFromRect()?

The reason I’m asking is because I need to know whether to:

  • Open a bug on bugzilla.
  • Open a feature request on bugzilla.
  • Stop using nsIDOMWindowUtils#nodesFromRect() in VimFx.
1 Like
#2

I tried to find out what it is supposed to do, but that was hopeless. What it actually does is very complicated internally, but at the end of the day quite simple and zero-dimension nodes don’t seem to be found. Might be intentional. This is basically a fat-finger detection feature and if the link is one one location (technically an undefined location) and the children elsewhere then perhaps it makes sense to only detect the children.

#3

Thanks for your answer @Lithopsian!

I tried to read the source code, too, and also found it difficult to understand.

In the end, I chose to get rid of nsIDOMWindowUtils#nodesFromRect(). Instead I use document.getElementsByTagName('*') and filter the elements using .getClientRects(), checking if any such rect is inside the viewport (the viewport is basically the arguments I passed to nsIDOMWindowUtils#nodesFromRect() previously.)

While the above might sound like a really slow solution, believe it or not but I can not notice any performance difference!

My conclusion is to avoid nsIDOMWindowUtils#nodesFromRect(). It is not that difficult to write yourself with the same performance, and most importantly, the exact behavior you’re looking for.

1 Like
#4

Other then noticing, did you try timing it with console.time and console.timeEnd?

#5

Yes.

#6

I know those numbers vary from system to system, but could you share what you got please?

#7

I did not create a specific test case or benchmark comparing the two techniques especially. Instead I measured the real-world performance of VimFx before and after.

If you press `f` using VimFx, every clickable element currently visible on the screen gets one or a few letters displayed on top of them. Type the letters for an element to simulate a click on it. The letters are called hints.

On "simple" pages the hints appear in less than 100ms, which is perceived as instant. On more "complicated" pages, notably Facebook, it can take more time. Some users have reported it can take about a second.

I added `console.time` just before starting to find elements that should get hints, and `console.timeEnd` just after the hints appear. This is the time span that is important to VimFx, so I didn't bother timing anything else.

My test then consisted of going to facebook.com, logging in and then scrolling five pages down. Then I displayed the hints a few times (by pressing `f`) and noted the results. After that, I switched from `nodesFromRect()` to the technique mentioned above and repeated the test.

At first I got slower results, but after reordering a few conditions in the code, resulting in the above technique, I got the same results.

On my computer, both before and after, showing the hints five pages down on Facebook takes 400ms ± 40ms.

I also use `elementFromPoint` when displaying hints, and as far as I remember, that's where most time is spent, not in `nodesFromRect` or its substitute.

#8

Its amazing what you can get done in JavaScript in a few ms, but don’t kid yourself that it is as fast as compiled C code using a fancy matrix manipulation algorithm. Restricting the list of nodes by tag name will give a big boost compared to filtering all nodes, but anything else you can think of to filter the list up front would be a help. Perhaps time just that search? If it is only a millisecond or two then that really is “fast enough”.

It is generally considered that anything that takes more than about 50ms will produce a noticeable jank in GUI smoothness, but responses to a user action can take far longer before it is perceived as unacceptable. 400ms is pushing it really, but it all depends on just what is expected from the user action and the user. If the time is being spent elsewhere anyway, perhaps look at that code and see if it can be tweaked. For example, getClientRects() is probably a lot slower than simply using the bounding rectangle.

#9

> It is generally considered that anything that takes more than about 50ms will produce a noticeable jank in GUI smoothness

Google believes the number is 100ms: https://developers.google.com/web/tools/chrome-devtools/profile/evaluate-performance/rail#response-respond-in-under-100ms

> 400ms is pushing it really, but it all depends on just what is expected from the user action and the user.

Yes, 400ms feels a bit slow. We _could_ filter out _less_ elements to make the hints appear faster, but then _more_ hints would be created. The time saved would then be more than lost when the user tries to find the correct hint in a sea of irrelevant hints. Also, the more hints the _longer_ hints, so users would also spend more time typing. This is a tricky trade-off.

> getClientRects() is probably a lot slower than simply using the bounding rectangle.

No. We used `getBoundingClientRect()` initially, but had to switch to `getClientRects()` to position the hint for line-wrapped links in a sane way. When doing so I was worried about performance, but my tests showed no difference. That might not be so strange, though, because many times `getClientRects()` returns a single rect, identical to the one returned by `getBoundingClientRect()`. And considering that `getBoundingClientRect()` can be "polyfilled" using `getClientRects()` , perhaps `getBoundingClientRect()` uses that internally. But enough guesswork and "probably:s" now.

> Restricting the list of nodes by tag name will give a big boost compared to filtering all nodes, but anything else you can think of to filter the list up front would be a help.

I can't restrict the nodes by tag because any type of element might be clickable, because of JavaScript. (Clickability is determined later by inspecting each element.)

I also found that checking if the elements are inside the viewport before filtering for clickability is significantly faster. It was before I reordered that I got the mentioned slowdown in my last comment.

> Its amazing what you can get done in JavaScript in a few ms, but don't kid yourself that it is as fast as compiled C code using a fancy matrix manipulation algorithm.

That's why I came here in the first place. I didn't want throw `nodesFromRect()` out of the window, because I was too thinking of the performance of C/C++, having access to things not exposed to JavaScript, etc.

When we first switched to `nodesFromRect` there was a significantly noticeable performance increase. However, that was some time ago. Looking back at the code with fresh eyes I realized that the same performance boost might be done with a slight code reorder. And as it turned out, I was right!

`nodesFromRect()` might be faster. I don't now. But it doesn't matter. Because in VimFx the difference isn't noticeable.

---

Thanks for all your responses! My immediate problem has been resolved. I'm still interested in knowing how `nodesFromRect` _is_ supposed to work though, purely because of curiousness :)

#10

Google’s 100ms applies to the response to a user action. That is pretty tight, and in practice you can easily delay two or three times that before it becomes annoying. Notice the next hint, though: always provide feedback for any action that takes longer than 500ms, so the user doesn’t think it failed and start randomly clicking stuff.

The 50ms is different: it is the maximum “busy loop” delay that will be noticed as an interruption to otherwise smooth GUI behaviour such as animations or scrolling. It applies to background operations that are not done in response to a specific user action. It corresponds roughly to human visual acuity which perceives 25 fps as continuous. Google recommends 16ms to correspond to a single 60 Hz frame, but that is really aiming for perfection.

If you respect the busy-loop limits, and preferably don’t slow down actual page rendering, you might be able to pre-calculate certain parameters for each page to respond more quickly to the user. You wouldn’t normally do this, because if every addon jumped on every page and started mapping it every which way, there’d be chaos, but sometimes it is worth it to be able to respond to the user near-instantly.