I've prepared a reproducer in a repository on github:
I don't understand hibernate internals enough to dig deeper, but this is clearly unwanted, because when I've just enabled it on our project, one of my API endpoints went from 10 queries to 816 queries and I gave up on examining the rest of them.
I've even tried to annotated all of toOne relations in our project with different combinations of @LazyToOne (with different LazyToOneOption) and @LazyGroup in the hopes that I've just misunderstood the feature, but it just went from broken (because of 100x times more queries which is unacceptable) to completely broken (where the data weren't even loading properly).
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)