I found a related topic but no idea:
I understand that this could be done using custom css, however, it would be great to be able to toggle this from the SF. This option is also preferred by on of our clients.
This is not just a visual option, since users get confused by clicking the lookup icons and expect to be able to select things, which does not work since the form is still in read mode.

There could be a checkbox to hide lookup icon in read mode on one or different levels:
- Theme
- Table
- Column
I would personally turn it off everywhere, since it confuses users.