KB484906: Fix the problem that security filter is not partially applied when generating Many-to-Many relations and sometimes leads to "Intermediate Row limit exceeds" error with Data Engine Version 2021

Starting with the release of Strategy ONE (March 2024), dossiers are also known as dashboards.
Description
In some cases, the security filter does not get applied when there are multiple filters defined in the security filter and only partial filter can be applied. Although there is no data leak since the security filter will be applied in later phase, it will impact the performance, sometimes even leads to "Intermediate Row limit exceeds" error. With the fix made under Data Engine Version 2021,  the security filter will be tried to apply partially to improve the performance.
Steps to Reproduce
This procedure is using Strategy Tutorial WH that is shipped with Strategy products.

1. Create Many to Many relation for Attribute MonthDE156959(Month), MOYDE156959(Month Of Year), QuarterDE156959(Quarter), RegionDE156959(Region):
   
1. MOYDE156959 -> MonthDE156959
2. QuarterDE156959 -> MonthDE156959
3. MOYDE156959 -> QuarterDE156959
4. RegionDE156959 -> MonthDE156959
2. Create User M20061R1DE with security filter defined as: MonthDE156959 in {20081, 20091, 20101} AND RegionDE156959 in {1}
3. Login Strategy Web as Security Filter user M20061R1DE
4. Click Add External Data -> Databases -> Select Tables, pick the following two tables and map Attribute Cust State Id, Month Id, Month Of Year to project attribute

1. Click Finish and choose Import as an In-memory Dataset, save as MTDI01

1. Click Add External Data -> Databases -> Select Tables, pick the following two tables and map Attribute Region Id, Month Id, Quarter Id to project attribute

1. Click Finish and choose Import as an In-memory Dataset, save as MTDI02
2. Create dossier with MTDI01 and MTDI02 as dataset, add into template: MOYDE156959 from MTDI01, QuarterDE156959 from MTDI02, set data source to be none

1. When viewing MCE Trace, we notice that the security filter does not get applied when generating Many to Many relation between Attribute MOYDE156959 and MonthDE156959. There is no data leak since the security filter will be applied in later phase, however it will impact the performance, sometimes even cause "Intermediate Row limit exceeds" error.

Tables Accessed:
Table5     [REL_MOYDE156959_MONTHDE156959]:          MOYDE156959, MonthDE156959,                         RELATIONSHIP_TABLE,                ONE_TO_MANY
select      [MonthDE156959]@[MONTH_ID],
                [MOYDE156959]@[MONTH_OF_YEAR]
from         MTDI01=MOY, Month
with Table Join Tree:             [REL_MOYDE156959_MONTHDE156959]
to             Ex0_tempcube9
Cause
This is a known issue in Strategy.
The security filter is not partly applied when generating Many to Many relations by using dataset relations.
Solution
This issue has been addressed in Strategy 2021.
To pick up the fix, upgrade to Strategy 2021 or above and enable project level setting "Data Engine Version" to 2021 or above. For more details on enabling the setting, refer to Change Data Engine Version
After the fix, we can see the security filter is partially applied when generating Many to Many relation between Attribute MOYDE156959 and MonthDE156959 in MCE Trace.
Tables Accessed:
Table5     [REL_MOYDE156959_MONTHDE156959]:          MOYDE156959, MonthDE156959,                         RELATIONSHIP_TABLE,                ONE_TO_MANY
select      [MonthDE156959]@[MONTH_ID],
                [MOYDE156959]@[MONTH_OF_YEAR]
from         MTDI01=MOY, Month
with Table Join Tree:             [REL_MOYDE156959_MONTHDE156959]<[MonthDE156959]@[MONTH_ID] in (200601)>
to             Ex0_tempcube9
 Strategy internal reference number for this technical note is DE158497.