With for example two top level course categories, when opening the the dropdown of the first category, and the cursor is hovering the first category or the dropdown, the checkbox of the second course category shows through the dropdown. In order to see this, the browser window has to be small enough so that the dropdown and checkbox actually overlap. (See the Attachment for an example.)

This also somewhat works for course subcategories. The exact rule seems to be, If you hover either the category with the currently open dropdown, or one of its parents, you could see the checkbox of every category, which is not a child of the currently hovered category.

This seems to be caused by MDL-70311, when bootstrap 4.6.0 added z-index: 1 for .custom-control.

Another place to look at would be https://github.com/moodle/moodle/blob/master/theme/boost/scss/bootstrap/_list-group.scss#L28. If I understand correctly, setting the z-index of every li.list-group-item-action puts all the child elements in their own stacking context. Because of that, the child elements of every .list-group-item-action only compare their z-index against each other and their parent element, but not against elements of other .list-group-item-action. That would explains why the checkboxes are shown, even though their z-index (1) is way less than the z-index of the dropdown (1000).