Ciel update

[hardhouse]

new logic for drugs

[mik-ohdsi]

Hi @hardhouse and @Alexdavv , I just noticed that from line 413 on there is still coding that is drug relevant (RxNorm, resolving MIN to IN etc.). Is all of this still relevant coding or should some of it be silenced? MINs now exist in the vocabulary, although only as non-standard, but I understand that this would be resolved in the generic update while the stage tables are being processed? relationships between SNOMED and RxNorm probably still make sense, but I thought they exist anyway independently from CIEL...
Other parts as of the comments:
-- add concept_relationships between NDFRT and RxNorm
-- add concept_relationships within SNOMED to decomponse multiple ingredients and map from procedure to drug
-- add concept_relationships within RxNorm from Ingredient to Ingredient
-- connect deprecated RxNorm ingredients to fresh ones by first word in concept_name
-- connect SNOMED ingredients to RxNorm by first word in concept_name

[mik-ohdsi]

Hi @hardhouse , I found a potential inconsistency where the coding seems to fail picking up the "preferred" concept name:

SELECT * FROM dev_ciel.concept_stage
 WHERE concept_code = '104048'; 

concept_id	concept_name	domain_id	vocabulary_id	concept_class_id	standard_concept	concept_code	valid_start_date	valid_end_date	invalid_reason
45953005	Gas-X with Maalox Extra Strength	Drug	CIEL	Drug		104048	2007-03-11	2099-12-31

SELECT * FROM dev_ciel.ciel_concept_name 
 WHERE concept_id = 104048
   AND locale = 'en'
   AND locale_preferred = 1;

concept_id	ciel_name	locale	creator	date_created	concept_name_id	voided	voided_by	date_voided	void_reason	uuid	concept_name_type	locale_preferred	date_changed	changed_by
104048	Calcium carbonate / simethicone	en	1	2007-03-11	9532	0	1			9532BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB	FULLY_SPECIFIED	1		1

Or maybe this is related to the not silenced code parts?

[hardhouse]

concept names come from p3

Vocabulary-v5.0/CIEL/load_stage.sql

Line 50 in a1f88f9

SELECT DISTINCT COALESCE(FIRST_VALUE(cn.ciel_name) OVER (

(just picks the longest name, without any checks)

[mik-ohdsi]

ah, okay. The section where we limit to locale = 'en' AND locale_preferred = 1 was meant to take care of that. (It seems I am quite lost in the code...)
Can you adjust?
And can you provide a statement about the remaining part of the code below line 413 that seems to do something with drugs. Do we need to keep it? Is it somewhat harmful / necessary / simply unnecessary?

[hardhouse]

1. committed
2. alas, i don’t know..

[Alexdavv]

@mik-ohdsi

I just noticed that from line 413 on there is still coding that is drug relevant (RxNorm, resolving MIN to IN etc.). Is all of this still relevant coding or should some of it be silenced? MINs now exist in the vocabulary, although only as non-standard, but I understand that this would be resolved in the generic update while the stage tables are being processed?

Generic update can't add the links, while it will definitely deprecate the links to non-standard. Some of the procedures can, but I still wouldn't stick to the logic that conflicts with the design. We add "Maps to" only to Standard concepts by the definition, while MINS are not. So yes, this code section seems to be needed.

relationships between SNOMED and RxNorm probably still make sense, but I thought they exist anyway independently from CIEL...

Don't we add them to the temporary table only in order to support the logic?

[mik-ohdsi]

sorry... warming this up once more...

Hi @hardhouse

We have some mismatches that I contribute to the logic that happens besides the existing CIEL-RxN + SNOMED mappings (as mentioned above). There are only a couple of those and they come from the DrugClass and MedSet CIEL domains:

concept_code	domain_id	concept_class_id	concept_name	invalid_reason	relationship_id	concept_id	vocabulary_id	concept_code	domain_id	concept_class_id	concept_name
161188	Drug	MedSet	H2-receptor antagonists	Maps to	19069019	RxNorm	5333	Drug	Ingredient	histamine
161201	Drug	MedSet	H2 receptor blockers	Maps to	19069019	RxNorm	5333	Drug	Ingredient	histamine
162299	Drug	Drug Class	ARBs (angiotensin II receptor blockers)	Maps to	963889	RxNorm	1999003	Drug	Ingredient	angiotensin II
162300	Drug	Drug Class	Aspirin-containing drug class	Maps to	1112807	RxNorm	1191	Drug	Ingredient	aspirin
162298	Drug	Drug Class	ACE inhibitors	Maps to	963889	RxNorm	1999003	Drug	Ingredient	angiotensin II
162674	Drug	Drug Class	Artemisinin-based Combination Therapy (ACT)	Maps to	1145680	RxNorm	2367409	Drug	Ingredient	artemisinin
5843	Drug	MedSet	Minerals iron supplements	Maps to	978577	RxNorm	577312	Drug	Ingredient	mineral oil, light
162305	Drug	Drug Class	Heparins	Maps to	1367571	RxNorm	5224	Drug	Ingredient	heparin
163351	Drug	Drug Class	Acetazolamide interacting drugs	Maps to	929435	RxNorm	167	Drug	Ingredient	acetazolamide

They do have mappings to SNOMED like this example shows:
SELECT * FROM dev_ciel.ciel_concept_reference_map
WHERE concept_id = 161188;
-- leading to:
SELECT * FROM dev_ciel.ciel_concept_reference_term
WHERE concept_reference_term_id IN ( 145447,218306 );

This provides us with the SNOMED code 108661002. But the mapping the logic had found is clearly wrong, as it seems to process some synonym containing the term "histamine" while it should be pointing at histamine-blocker (the opposite).

As there are only very few correct entries from these domains, we might want to ignore them altogether, because they do not give us additional value. Can you find out where in the code we could exclude mapping those domains Drug Class and MedSet?

[hardhouse]

@mik-ohdsi hi, just add these relationships to manual table (dev_ciel.concept_relationship_manual) as 'D', and the procedure ProcessManualRelationships() will do the rest

[mik-ohdsi]

Hi @hardhouse - I was almost certain I had created a comment here a while ago... but none to see...
What I wanted to ask: can we exclude those CIEL Drug Class and MedSet classes for good (they are of little or no use in the CDM anyway), maybe like in the initial select so that we ignore them for the rest of the processing? I would not like to proceed with the exclusion by manual table as we would have to curate that in the future as well (and this is bound to be forgotten).
thanks ~ mik

[hardhouse]

Hi @mik-ohdsi
yes (if i understood correctly), need to exclude these codes in corresponding query and manually deprecate these relationships in the base tables

[mik-ohdsi]

@hardhouse - I would hope there are no relationships created if we do not have a source code for it. We simply want to suppress all "source codes" of the classes Drug Class and MedSet. Maybe by excluding them from the "INSERT INTO concept_stage" ?

[hardhouse]

no, that's not how it works

[mik-ohdsi]

wait, are you telling me we would create relationships in concept_relationship_stage for concepts that are not in concept_stage? It sounds weird to me that we would do that... well, only for the case that a relationship existed before and we want to remove it (which we added to the logic, if I am not mistaken)... now I am a little confused...

[hardhouse]

hmm, it seems to me that there are a number of issues that need to be clarified:)

1. there is absolutely no problem creating/changing relationships for concepts that are not in the concept_stage (but they must exists in the concept table), because this is a normal situation, the generic can work with it
2. "we would create relationships in concept_relationship_stage" why? as far as i undestood, these relationships are created by some of our code in the load_stage, right? so in this case we need to add an additional condition like "and (concept_code_1, concept_code_2) <> ('aaa','bbb')" etc. but, again, this is not the "right" way, especially for 'Maps to', because will need to manually edit the base table and mentally calculate all other dependent (if they exist) relationships