Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cohort Definition Validation Feature API #1929

Draft
wants to merge 60 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 56 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
5464709
Added Annotation Code
jduke99 Feb 1, 2021
6735945
updating annotation code to test build.
Mar 2, 2021
f1203ec
New Annotation Tables
jduke99 Mar 31, 2021
704b5a2
Updated Annotation
jduke99 Apr 1, 2021
8e4fc91
Update CohortSample.java
jduke99 May 8, 2021
de445db
annotion link button now works
May 18, 2021
18319f8
removing results from Annotation.java
May 19, 2021
b6d5b07
trying to make saving results to annotation_results table work
May 27, 2021
b82ef09
working annotation_results saving
May 27, 2021
65028fc
initial commit for ResultController and ResultService
Jun 2, 2021
7b02d10
can now query for all results of a certain annotation ID
Jun 3, 2021
45ffb5f
added todo in AnnotationController
Jun 3, 2021
b2166a8
default annotation GET now returns summary instead of full question e…
Jun 15, 2021
7b7ce8e
removed some printlines, also added todo about performance
Jun 15, 2021
d769a68
moving to prepared statements for annotation_results
Jul 29, 2021
b002dfe
working insertion and 'updating'. Tomorrow will change all QuestionID…
Jul 29, 2021
348e67c
working results by Question Set ID, albiet slow
Jul 30, 2021
1326f9a
querying for results based on QuestionSetId now is much faster
Jul 30, 2021
791e3f9
adding in orderered params to all jdbcTemplate queries
Jul 30, 2021
ba72b44
removed all instances/mentions of long and changed to int. Also fixed…
Jul 30, 2021
c695f24
working getSets query
Aug 3, 2021
1afa631
creating annotation study table to aid in query speed for results sup…
Aug 11, 2021
e5b1e2a
more work done on supertable- everything should be using annotation_s…
Aug 12, 2021
92271b4
return plain Results from the getResults endpoint
Aug 12, 2021
3329bde
changing sampleName to cohortSampleId in AnnotationController
Aug 13, 2021
b8b61f4
changing sampleName to cohortSampleId in AnnotationController pt 2
Aug 13, 2021
734b8b3
complete results now querying by sampleId instead of Study ID when th…
Aug 14, 2021
47c6b31
finding samples and sets by cohort ID now uses inner join instead of …
Aug 15, 2021
37542bf
adding answer text in additon to answer value to the super table
Aug 17, 2021
6bd6d66
setting AnswerValue now
Aug 17, 2021
abca93c
using LinkedHashSet for QuestionSet to ensure we add questions in order
Aug 18, 2021
c880d3d
can now delete questionSets
Aug 19, 2021
18a4d02
Adding orderby for questionSet Question retrieval
Aug 20, 2021
11f9df0
checking to delete Results by annotation only instead of using questi…
Aug 21, 2021
c793786
getting answer value from result instead of answer for super table
Aug 21, 2021
a0b4bd3
Merge branch 'master' of https://github.com/OHDSI/WebAPI into annotat…
charhart Aug 22, 2021
514673b
merge master into annotation feature; rename flyway to reflect latest…
charhart Aug 22, 2021
fed83bd
Merge branch 'annotation2.0' of https://github.com/cahilton/WebAPI in…
charhart Aug 22, 2021
1bbf4dd
removing todo from nav service to navigate by sample name- this will …
Aug 22, 2021
dd28f06
removing unused result link in Annotation entity
Aug 22, 2021
4f08ca2
removing unused sample_name field from annotation
Aug 22, 2021
c375503
removing unused getters and setters in Annotation Entity
Aug 22, 2021
d0dfb87
removing unused imports from Annotation Entity
Aug 22, 2021
f0421a1
removing unused code in AnnotationController, also removing only-for-…
Aug 22, 2021
712c388
removing todo for searching for annotation by annotationID instead of…
Aug 22, 2021
8854ba4
removing unused imports from AnnotationController and AnnotationServi…
Aug 22, 2021
d58f51c
changing AnnotationRepository to use base findById instead of explici…
Aug 22, 2021
9c558ec
updated AnswerRepository to use default findById method instead of query
Aug 22, 2021
cecc37a
updated QuestionRepository to use default findById method instead of …
Aug 22, 2021
26eff09
removing all chaff from Annotation Result package- unused fields etc
Aug 22, 2021
075af89
updated QuestionSet to use default findById, removed unused imports a…
Aug 22, 2021
cdca276
removing more unused imports from QuestionSet repository
Aug 22, 2021
530fa59
using sequence for annotation_id instead of generated value
Aug 22, 2021
2643f96
using generator for all IDs in Question, QuestionSet, Answer
Aug 22, 2021
45057a4
rename annotation sequences
charhart Aug 23, 2021
aa9f441
corrected sequence names for id generators of entities. Also removed …
Aug 23, 2021
41058f3
Merge branch 'OHDSI:master' into annotation2.0-cknoll
chrisknoll Oct 20, 2021
5a4905f
Merge remote-tracking branch 'remotes/upstream/master' into annotatio…
chrisknoll Nov 8, 2021
8814282
Added security migration scripts.
chrisknoll Nov 10, 2021
37d1c60
Merge pull request #1 from cahilton/annotation2.0-cknoll
charhart Nov 16, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions src/main/java/org/ohdsi/webapi/annotation/annotation/Annotation.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
package org.ohdsi.webapi.annotation.annotation;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.ManyToOne;
import javax.persistence.JoinColumn;
import javax.persistence.Id;
import javax.persistence.Table;

import org.hibernate.annotations.GenericGenerator;
import org.hibernate.annotations.Parameter;
import org.ohdsi.webapi.annotation.set.QuestionSet;

@Entity(name = "Annotation")
@Table(
name = "annotation"
)
public class Annotation {

@Id
@GenericGenerator(
name="annotation_generator",
strategy = "org.hibernate.id.enhanced.SequenceStyleGenerator",
parameters = {
@Parameter(name="sequence_name",value="annotation_seq"),
@Parameter(name="increment_size",value="1")
}
)
@GeneratedValue(generator = "annotation_generator")
@Column(name = "annotation_id")
private int id;

@Column(name = "subject_id")
private int subjectId;

@Column(name = "cohort_sample_id")
private int cohortSampleId;

@ManyToOne
@JoinColumn(name = "question_set_id")
private QuestionSet questionSet;

//***** GETTERS/SETTERS ******

/**
* @return the id
*/
public int getId() {
return id;
}

/**
* @param id the id to set
*/
public void setId(int id) {
this.id = id;
}

/**
* @return the subjectId
*/
public int getSubjectId() {
return subjectId;
}

/**
* @param subjectId the subjectId to set
*/
public void setSubjectId(int subjectId) {
this.subjectId = subjectId;
}

/**
* @return the cohortId
*/
public int getCohortSampleId() {
return cohortSampleId;
}

/**
* @param cohortSampleId the cohortId to set
*/
public void setCohortSampleId(int cohortSampleId) {
this.cohortSampleId = cohortSampleId;
}

/**
* @return the set
*/
public QuestionSet getQuestionSet() {
return questionSet;
}

/**
* @param set the set to set
*/
public void setQuestionSet(QuestionSet questionSet) {
this.questionSet = questionSet;
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
package org.ohdsi.webapi.annotation.annotation;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there a namespace 'annotation.annotation'?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in this case- The overall functionality we're adding gets called "annotation", and inside that overall annotation idea we have the true Annotation package. Not sure what the best fix is here if it needs changed, but my take is that it's fine.


import java.util.ArrayList;
import java.util.List;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Consumes;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.QueryParam;
import javax.ws.rs.PathParam;
import org.json.JSONObject;
import org.json.JSONArray;
import org.ohdsi.webapi.annotation.result.ResultService;
import org.ohdsi.webapi.annotation.study.Study;
import org.ohdsi.webapi.annotation.study.StudyService;
import org.ohdsi.webapi.cohortdefinition.CohortDefinition;
import org.ohdsi.webapi.cohortdefinition.CohortDefinitionRepository;
import org.ohdsi.webapi.cohortsample.CohortSample;
import org.ohdsi.webapi.cohortsample.CohortSampleRepository;
import org.ohdsi.webapi.cohortsample.CohortSamplingService;
import org.ohdsi.webapi.cohortsample.dto.SampleElementDTO;
import org.ohdsi.webapi.source.Source;
import org.ohdsi.webapi.source.SourceService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.ohdsi.webapi.annotation.set.QuestionSetRepository;
import org.ohdsi.webapi.annotation.set.QuestionSet;
import org.springframework.web.bind.annotation.RequestBody;


@Path("annotations")
@Component
public class AnnotationController {

@Autowired
private StudyService studyService;

@Autowired
private ResultService resultService;

@Autowired
private CohortDefinitionRepository cohortDefinitionRepository;

@Autowired
private CohortSamplingService cohortSamplingService;

@Autowired
private CohortSampleRepository sampleRepository;

@Autowired
private AnnotationService annotationService;

@Autowired
private QuestionSetRepository questionSetRepository;

@Autowired
private SourceService sourceService;

@GET
@Path("/")
@Produces(MediaType.APPLICATION_JSON)
public List<AnnotationSummary> getAnnotations(
@QueryParam("cohortSampleId") final int cohortSampleId,
@QueryParam("subjectId") final int subjectId,
@QueryParam("setId") final int setId
) {
List<Annotation> returnAnnotations;
returnAnnotations = getFullAnnotations(cohortSampleId,subjectId,setId);
List<AnnotationSummary> summaries = new ArrayList();
for(Annotation singleAnno : returnAnnotations){
// can we see about doing this in a more performant manner?
AnnotationSummary tempAnnoSummary=new AnnotationSummary(singleAnno);
summaries.add(tempAnnoSummary);
}
return summaries;
}

@GET
@Path("/fullquestion")
@Produces(MediaType.APPLICATION_JSON)
public List<Annotation> getFullAnnotations(
@QueryParam("cohortSampleId") final int cohortSampleId,
@QueryParam("subjectId") final int subjectId,
@QueryParam("setId") final int setId
) {
List<Annotation> returnAnnotations=null;
if (cohortSampleId != 0 && subjectId != 0 && setId != 0) {
System.out.println("made it into the search function");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to find and remove all System.out calls and move to logging or remove entirely.

returnAnnotations= annotationService.getAnnotationByCohortSampleIdAndBySubjectIdAndByQuestionSetId(cohortSampleId, subjectId, setId);
}
else if (cohortSampleId !=0 && setId!=0){
returnAnnotations= annotationService.getAnnotationByCohortSampleIdAndByQuestionSetId(cohortSampleId,setId);
}
else if(cohortSampleId!=0){
returnAnnotations= annotationService.getAnnotationsByCohortSampleId(cohortSampleId);
}
else if(setId !=0){
returnAnnotations= annotationService.getAnnotationsByQuestionSetId(setId);
}
else{
returnAnnotations=annotationService.getAnnotations();
}
return returnAnnotations;
}

@POST
@Path("/")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public void addResult(@RequestBody String payload) {
System.out.println(payload);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't usually parse raw JSON text from the POST. Instead, you define a class that represents the data structure, and the service will automatically serialize the request body into the JSON form.

In addition, we shouldn't leave 'System.out.println' in the sourcecode. Use a logger.

JSONObject jsonpayload = new JSONObject(payload);
System.out.println(jsonpayload);
System.out.println(jsonpayload.get("results"));
System.out.printf("cohortId: %s\n",jsonpayload.get("cohortId").toString());
System.out.printf("cohortSampleId: %s\n",jsonpayload.get("cohortSampleId").toString());
System.out.printf("subjectId: %s\n",jsonpayload.get("subjectId").toString());
System.out.printf("setId: %s\n",jsonpayload.get("setId").toString());
Annotation tempAnnotation = annotationService.getAnnotationsByAnnotationId(jsonpayload.getInt("id"));
System.out.printf("annotationID:%d\n",tempAnnotation.getId());
JSONArray array = jsonpayload.getJSONArray("results");
Study study = studyService.getStudyByQuestionSetIdAndSampleId(jsonpayload.getInt("setId"),jsonpayload.getInt("cohortSampleId") );
resultService.insertResults(tempAnnotation,array,study);
}

@POST
@Path("/sample")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public void addAnnotation(@RequestBody String payload) {
JSONObject jsonpayload = new JSONObject(payload);
int cohortSampleId = Integer.parseInt(jsonpayload.get("sampleId").toString());
CohortSample cohortSample =sampleRepository.findById(cohortSampleId);
List<SampleElementDTO> temp = cohortSamplingService.getSample(cohortSampleId, false).getElements();
Study study = new Study();
int questionSetId = Integer.parseInt(jsonpayload.get("annotationSetId").toString());
QuestionSet questionSet = questionSetRepository.findById(questionSetId);
study.setQuestionSet(questionSetRepository.findById(questionSetId));
study.setCohortSample(cohortSample);
CohortDefinition cohortDefinition= cohortDefinitionRepository.findOneWithDetail(cohortSample.getCohortDefinitionId());
study.setCohortDefinition(cohortDefinition);
Source source = sourceService.findBySourceKey(jsonpayload.get("sourceKey").toString());
study.setSource(source);
studyService.addStudy(study);
for (SampleElementDTO element : temp){
System.out.println("element"+element);
System.out.println("element GetPersonID"+element.getPersonId());
Annotation annotation = new Annotation();
annotation.setSubjectId(Integer.parseInt(element.getPersonId()));
annotation.setCohortSampleId(cohortSampleId);
annotation.setQuestionSet(questionSet);
annotationService.addAnnotation(annotation);
}
}

@GET
@Path("/{annotationID}")
@Produces(MediaType.APPLICATION_JSON)
public Annotation getResults(@PathParam("annotationID") String annotationID) {
int annotationIdInt=Integer.parseInt(annotationID);
Annotation ourAnno =annotationService.getAnnotationsByAnnotationId(annotationIdInt);
return ourAnno;
}
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
package org.ohdsi.webapi.annotation.annotation;

import java.util.HashSet;
import java.util.Set;
import org.ohdsi.webapi.annotation.result.ResultDto;

public class AnnotationDto {

private int id;
private int subjectId;
private int sampleId;
private int annotationSetId;
private String name;
private Set<ResultDto> results = new HashSet();

//***** GETTERS/SETTERS ******

/**
* @return the id
*/
public int getId() {
return id;
}

/**
* @param id the id to set
*/
public void setId(int id) {
this.id = id;
}

/**
* @return the subjectId
*/
public int getSubjectId() {
return subjectId;
}

/**
* @param subjectId the subjectId to set
*/
public void setSubjectId(int subjectId) {
this.subjectId = subjectId;
}

/**
* @return the cohortId
*/
public int getsampleId() {
return sampleId;
}

/**
* @param sampleId the cohortId to set
*/
public void setsampleId(int sampleId) {
this.sampleId = sampleId;
}

/**
* @return the setId
*/
public int getannotationSetId() {
return annotationSetId;
}

/**
* @param setId the setId to set
*/
public void setannotationSetId(int setId) {
this.annotationSetId = setId;
}

/**
* @return the results
*/
public Set<ResultDto> getResults() {
return new HashSet<ResultDto>(results);
}

/**
* @param results the results to set
*/
protected void setResults(Set<ResultDto> results) {
this.results = results;
}

/**
* @return the sample name
*/
public String getSampleName() {
return name;
}

/**
* @param name to the slected sample name
*/
public void setSampleName(String name) {
this.name = name;
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package org.ohdsi.webapi.annotation.annotation;

import org.springframework.data.jpa.repository.JpaRepository;
import java.util.List;
import java.util.Set;
import org.ohdsi.webapi.annotation.annotation.Annotation;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;

public interface AnnotationRepository extends JpaRepository<Annotation, Integer> {

public Set<Annotation> findOneByCohortSampleIdAndSubjectIdAndQuestionSetId(int cohortSampleId, int subjectId, int questionSetId);
public List<Annotation> findByCohortSampleIdAndQuestionSetId(int cohortSampleId, int questionSetId);
public List<Annotation> findByCohortSampleId(int cohortSampleId);
public List<Annotation> findByQuestionSetId(int questionSetId);
public Annotation findById(int annotation_id);
}
Loading