Hi HJ,

It could be that the hidden fields are still needed for operations such as conditional layouts. The GUI needs that information to color the background or set a font. Could that be the case? 

Hi Mark,
There is no conditional layout defined on that table and at least some of the columns are also not part of prefilters.

Hendrik-Jan
 

Talked to one of the devs, this is indeed the case. When synchronizing to IAM, the effective rights per column is calculated. If not unauthorized, the column is syncrhonized to IAM. Whenever data is then selected from a table, all columns present in IAM will be included in the API unless a $select= is specified.

Of course this is based on role rights so nobody sees more columns/data than they are allowed to, but if you have rights to the hidden column, the data will be included in the API request if the $select= is not used.

Reason being is that the underlying interface views will not recalculate whether or not the column should be included in the request as this would cost a lot of performance. For every column, we would have to first check the rights (already done), gather the maximum access type (already done), then look if the column is used in for instance a conditional layout condition, or in a prefilter or, or, or… That decreases the performance significantly.

So if you truly want to prevent the column/data from occurring in the API call, a View (with appropriate rights) would indeed solve that.

Just as an additional remark to Mark’s reply.

Hidden fields do effect the build-up of the UI in Universal, even when they are invisible, and is in our case used to achieve some specific layout in a form.

Although I don’t consider this a bug, some wise(?) developer once said something like: “When a bug exists long enough, it will become someone’s feature and should therefore no longer be considered a bug”. 

We are encountering performance problems due to this design decision, which is interesting considering this is done because of performance concerns.

We have pretty wide views and tables with many lookups, the grids however are very limited in what columns are shown. Because of this there is ALOT more load on both the SQL engine (normally the query engine will optimize its execution plan and only retrieve what you need, including joins etc) and a lot more data transferred to the client.

Is there are chance this design decision can be reconsidered?