Consider active flag in simple refset valid keys assertion

[dionmcm]

This is a test that is failing in the Australian edition. There are 30 rows in the history of the Australian edition where a concept has

• been in a reference set with member id 1
• removed by member id 1 being inactivated
• been added back to the reference set with a new member id 2
• some have subsequently been removed again with member id 2 inactivated

That results in 2 rows in the snapshot where the refsetid and referencedcomponentid are the same, but at any given timepoints one is active and the other inactive (or nonexistent).

Here's an example from our release

94ff4357-bf3f-48ea-87b5-510b02475611	20120531	0	32506021000036107	32570111000036109	69630006
4f8cf7be-6c69-4702-a486-1ec3f3a3643a	20121130	1	32506021000036107	32570111000036109	69630006

There's nothing (I'm aware of) in the RF2 spec that makes this invalid, although I can see from the point of view of maintaining a compact snapshot it is desirable to reactivate the original refset member in this scenario rather than create a new member. That is also our practice for those reasons, and these are historical anomalies.

This suggestion is to add in the active field to account for this scenario. I'm not sure if there's an alternative with the RVF to instead whitelist for our release the affected historical rows?

Unfortunately this isn't something that can be "fixed" in the release anymore, even if we were to retire the new membership and reactivate the original one we'd still have two rows referencing he same concept for the same reference set id.

[dmcgihtsdo]

Thanks for the pull request @dionmcm.

I agree the historical issues should be ignored, but I think your fix will also ignore current-cycle / new duplicates.

Continuing your examples, let's pretend the next version is actually 20121130. In this situation, we want to flag up the error but ignore all historical issues. I have written up the query below, but it has went from ~3 seconds to ~13 seconds.

SELECT a.refsetid, a.referencedcomponentid, COUNT(a.id) AS count FROM simplerefset_s a INNER JOIN ( SELECT referencedcomponentid, refsetid, effectivetime FROM simplerefset_s WHERE effectivetime = 20121130 OR(active = 1 AND effectivetime != 20121130) GROUP BY referencedcomponentid, refsetid, effectivetime) b ON a.referencedcomponentid = b.referencedcomponentid AND a.refsetid = b.refsetid GROUP BY a.refsetid, a.referencedcomponentid HAVING COUNT(a.id) > 1;

In the RVF template syntax, I think 20121130 would be replaced with <VERSION>.