Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata uploader batch endpoints #1408

Merged
merged 60 commits into from
Jan 10, 2023
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
2e25abb
starting to create new batch api endpoints
ksierks Nov 4, 2022
c186502
Merge branch 'metadata-uploader-base' into metadata-uploader-batch-en…
ksierks Nov 4, 2022
9f9a7f7
sending requests in chunks
ksierks Nov 9, 2022
6aa9d6c
fixing error handling
ksierks Nov 9, 2022
bdec14e
Merge branch 'metadata-uploader-base' into metadata-uploader-batch-en…
ksierks Nov 10, 2022
1ee66b0
starting to add unique key on label for metadata_field table
ksierks Nov 15, 2022
23aaf61
Merge branch 'metadata-uploader-base' into metadata-uploader-batch-en…
ksierks Nov 23, 2022
6e72446
adding promise chaining back in
ksierks Nov 25, 2022
211152d
moving returning ResponseEntity into controller
ksierks Dec 5, 2022
05b1340
fixing broken test
ksierks Dec 5, 2022
d9bb07f
fixing create project sample from modal
ksierks Dec 5, 2022
8a53a63
moving returning ResponseEntity into controller
ksierks Dec 5, 2022
60e4bec
adding more tests
ksierks Dec 5, 2022
8845722
adding comments
ksierks Dec 5, 2022
bb75d78
Merge branch 'metadata-uploader-base' into metadata-uploader-batch-en…
ksierks Dec 5, 2022
d9f98a8
fixing unmounted component error
ksierks Dec 5, 2022
79d4c90
changing path names
ksierks Dec 7, 2022
b357be2
returning a mult-status response from controller
ksierks Dec 7, 2022
43780af
adding functionality to calculate chunk size
ksierks Dec 7, 2022
a059c96
creating CreateSampleResponse and UpdateSampleResponse objects
ksierks Dec 8, 2022
c162ee4
Merge branch 'metadata-uploader-base' into metadata-uploader-batch-en…
ksierks Dec 8, 2022
983aa63
fixing tests
ksierks Dec 8, 2022
3576845
playing with a second web driver to have seperate users logged in sim…
ksierks Dec 9, 2022
2993915
cleaning up unused, old code
ksierks Dec 9, 2022
2f65a56
setting up delete samples during import test
ksierks Dec 12, 2022
07a587b
setting up create new sample during import test
ksierks Dec 12, 2022
e2c6c7f
setting up remove access during import test
ksierks Dec 12, 2022
ac0fdaa
finishing tests
ksierks Dec 16, 2022
4a394f0
updating documentation
ksierks Dec 16, 2022
ad886b2
Merge branch 'metadata-uploader-base' into metadata-uploader-batch-en…
ksierks Dec 16, 2022
deca056
trying to fix flaky tests
ksierks Dec 19, 2022
84c31f2
trying to fix flaky tests
ksierks Dec 19, 2022
874f6ab
trying to fix flaky tests
ksierks Dec 20, 2022
eb581b8
removing old restriction code
ksierks Dec 21, 2022
d0b3495
adding more comments
ksierks Dec 21, 2022
4712e69
using metadataSaveDetails from state
ksierks Dec 21, 2022
3eda36c
removing log statements
ksierks Dec 21, 2022
8f6bde8
adding partial success upload notification
ksierks Dec 21, 2022
8665d0c
removing returned metadataSaveDetails as it is redundant
ksierks Dec 21, 2022
687f4bf
creating a import-utilities.js file for refactoring
ksierks Dec 21, 2022
6ee2590
simplifying code and calling Promise.all
ksierks Dec 21, 2022
0d84b2c
using forEach instead of map
ksierks Dec 21, 2022
54cc39e
returning more explicit filter
ksierks Dec 21, 2022
03e22a8
adding comments
ksierks Dec 22, 2022
9071f0c
more refactoring
ksierks Dec 22, 2022
2cd44f6
more refactoring
ksierks Dec 23, 2022
63ca764
fixing some tests
ksierks Jan 4, 2023
5d7b998
Merge branch 'metadata-uploader-base' into metadata-uploader-batch-en…
ksierks Jan 6, 2023
47a06aa
fixing chunkArray function
ksierks Jan 6, 2023
234c460
fixing chunkArray function
ksierks Jan 6, 2023
a62081d
taking care of all the 'any' types
ksierks Jan 6, 2023
b7cbb89
memorizing filter
ksierks Jan 6, 2023
600dcbd
applying eslint suggestions
ksierks Jan 6, 2023
2a6ad22
making a clone of param because splice is destructive
ksierks Jan 6, 2023
c391d57
resolving typescript warning
ksierks Jan 6, 2023
3ce8e16
resolving typescript warning
ksierks Jan 6, 2023
8a388a5
renaming type
ksierks Jan 6, 2023
7931412
fixing tests
ksierks Jan 7, 2023
e17ee8d
applying eslint suggestion
ksierks Jan 9, 2023
7fd0d33
renaming SampleItemResponse
ksierks Jan 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,19 +1,22 @@
package ca.corefacility.bioinformatics.irida.model.sample;

import ca.corefacility.bioinformatics.irida.model.sample.metadata.MetadataEntry;
import org.hibernate.envers.Audited;
import org.springframework.data.jpa.domain.support.AuditingEntityListener;
import java.util.List;
import java.util.Objects;

import javax.persistence.*;
import javax.validation.constraints.NotNull;
import java.util.List;
import java.util.Objects;

import org.hibernate.envers.Audited;
import org.springframework.data.jpa.domain.support.AuditingEntityListener;

import ca.corefacility.bioinformatics.irida.model.sample.metadata.MetadataEntry;

/**
* Describes an individual field in a {@link MetadataTemplate}.
*/
@Entity
@Table(name = "metadata_field")
@Table(name = "metadata_field",
uniqueConstraints = @UniqueConstraint(columnNames = { "label" }, name = "UK_METADATA_FIELD_LABEL"))
@Audited
@EntityListeners(AuditingEntityListener.class)
public class MetadataTemplateField {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,19 @@
* UI Request to update an existing sample
*/
public class UpdateSampleRequest extends CreateSampleRequest {
public UpdateSampleRequest(String name, String organism, String description, List<MetadataEntryModel> metadata) {
private Long sampleId;

public UpdateSampleRequest(Long sampleID, String name, String organism, String description,
List<MetadataEntryModel> metadata) {
super(name, organism, description, metadata);
this.sampleId = sampleID;
}

public Long getSampleId() {
return sampleId;
}

public void setSampleId(Long sampleId) {
this.sampleId = sampleId;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,9 @@

import ca.corefacility.bioinformatics.irida.model.sample.Sample;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.*;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.ajax.AjaxErrorResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.ajax.AjaxResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.ajax.AjaxSuccessResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.ajax.*;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.projects.dto.ValidateSampleNamesRequest;
import ca.corefacility.bioinformatics.irida.ria.web.exceptions.UIMetadataImportException;
import ca.corefacility.bioinformatics.irida.ria.web.exceptions.UIShareSamplesException;
import ca.corefacility.bioinformatics.irida.ria.web.models.tables.AntTableResponse;
import ca.corefacility.bioinformatics.irida.ria.web.projects.dto.DownloadRequest;
Expand Down Expand Up @@ -62,31 +61,40 @@ public ResponseEntity<SampleNameValidationResponse> validateNewSampleName(@Reque
}

/**
* Create a new sample within a project
* Create new samples within a project
*
* @param request Details about the sample
* @param requests Details about the samples
* @param projectId current project identifier
* @param locale current users locale
* @return result of creating the project
* @return result of creating the samples
*/
@PostMapping("/add-sample")
deepsidhu85 marked this conversation as resolved.
Show resolved Hide resolved
public ResponseEntity<AjaxResponse> createSampleInProject(@RequestBody CreateSampleRequest request,
public ResponseEntity<AjaxResponse> createSamplesInProject(@RequestBody CreateSampleRequest[] requests,
@PathVariable long projectId, Locale locale) {
return uiProjectSampleService.createSample(request, projectId, locale);
try {
return ResponseEntity.ok(new AjaxUpdateItemSuccessResponse(
uiProjectSampleService.createSamples(requests, projectId, locale)));
} catch (UIMetadataImportException e) {
return ResponseEntity.status(HttpStatus.CONFLICT).body(new AjaxFormErrorResponse(e.getErrors()));
}
}

/**
* Update a sample within a project
* Update samples within a project
*
* @param request Details about the sample
* @param sampleId sample identifier
* @param requests Details about the samples
* @param locale current users locale
* @return result of creating the project
* @return result of updating the samples
*/
@PatchMapping("/add-sample/{sampleId}")
public ResponseEntity<AjaxResponse> updateSampleInProject(@RequestBody UpdateSampleRequest request,
@PathVariable Long projectId, @PathVariable long sampleId, Locale locale) {
return uiProjectSampleService.updateSample(request, sampleId, locale);
@PatchMapping("/add-sample")
deepsidhu85 marked this conversation as resolved.
Show resolved Hide resolved
public ResponseEntity<AjaxResponse> updateSamplesInProject(@RequestBody UpdateSampleRequest[] requests,
Locale locale) {
try {
return ResponseEntity.ok(
new AjaxUpdateItemSuccessResponse(uiProjectSampleService.updateSamples(requests, locale)));
} catch (UIMetadataImportException e) {
return ResponseEntity.status(HttpStatus.CONFLICT).body(new AjaxFormErrorResponse(e.getErrors()));
}
}

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
package ca.corefacility.bioinformatics.irida.ria.web.exceptions;

import java.util.Map;

/**
* Exception to be thrown by the UI when there are errors during a metadata import.
*/
public class UIMetadataImportException extends Exception {
private final Map<String, String> errors;

public UIMetadataImportException(Map<String, String> errors) {
super("Metadata Import Exception");
this.errors = errors;
}

public Map<String, String> getErrors() {
return this.errors;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
import java.util.*;
import java.util.stream.Collectors;

import javax.validation.ConstraintViolationException;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.MessageSource;
import org.springframework.http.HttpStatus;
Expand All @@ -20,14 +22,11 @@
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.LockedSamplesResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.SampleNameValidationResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.UpdateSampleRequest;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.ajax.AjaxCreateItemSuccessResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.ajax.AjaxErrorResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.ajax.AjaxResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.dto.ajax.AjaxUpdateItemSuccessResponse;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.projects.dto.MetadataEntryModel;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.projects.dto.ValidateSampleNameModel;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.projects.dto.ValidateSampleNamesRequest;
import ca.corefacility.bioinformatics.irida.ria.web.ajax.projects.dto.ValidateSampleNamesResponse;
import ca.corefacility.bioinformatics.irida.ria.web.exceptions.UIMetadataImportException;
import ca.corefacility.bioinformatics.irida.service.ProjectService;
import ca.corefacility.bioinformatics.irida.service.sample.MetadataTemplateService;
import ca.corefacility.bioinformatics.irida.service.sample.SampleService;
Expand Down Expand Up @@ -123,65 +122,107 @@ public ResponseEntity<SampleNameValidationResponse> validateNewSampleName(String
}

/**
* Create a new sample in a project
* Create new samples in a project
*
* @param request {@link CreateSampleRequest} details about the sample to create
* @param requests Each {@link CreateSampleRequest} contains details about the sample to create
* @param projectId Identifier for the current project
* @param locale Users current locale
* @return result of creating the sample
* @throws UIMetadataImportException if there are errors creating the project samples
*/
@Transactional
public ResponseEntity<AjaxResponse> createSample(CreateSampleRequest request, Long projectId, Locale locale) {
try {
Project project = projectService.read(projectId);
Sample sample = new Sample(request.getName());
if (!Strings.isNullOrEmpty(request.getOrganism())) {
sample.setOrganism(request.getOrganism());
public String createSamples(CreateSampleRequest[] requests, Long projectId, Locale locale)
throws UIMetadataImportException {
Map<String, String> errors = new HashMap<>();
for (CreateSampleRequest request : requests) {
try {
createSample(projectId, request);
} catch (Exception e) {
errors.put(request.getName(), e.getMessage());
}
if (!Strings.isNullOrEmpty(request.getDescription())) {
sample.setDescription(request.getDescription());
}
Join<Project, Sample> join = projectService.addSampleToProjectWithoutEvent(project, sample, true);
if (request.getMetadata() != null) {
Set<MetadataEntry> metadataEntrySet = createMetadata(request.getMetadata());
sampleService.updateSampleMetadata(sample, metadataEntrySet);
}
if (errors.isEmpty()) {
return messageSource.getMessage("server.AddSample.success", null, locale);
} else {
throw new UIMetadataImportException(errors);
}
}

/**
* Create a new sample in a project
*
* @param request {@link CreateSampleRequest} contains details about the sample to create
* @param projectId Identifier for the current project
* @return result of creating the sample
* @throws EntityNotFoundException if the identifier does not exist in the database
*/
@Transactional
public Long createSample(Long projectId, CreateSampleRequest request) throws EntityNotFoundException {
Project project = projectService.read(projectId);
Sample sample = new Sample(request.getName());
if (!Strings.isNullOrEmpty(request.getOrganism())) {
sample.setOrganism(request.getOrganism());
}
if (!Strings.isNullOrEmpty(request.getDescription())) {
sample.setDescription(request.getDescription());
}
Join<Project, Sample> join = projectService.addSampleToProjectWithoutEvent(project, sample, true);
if (request.getMetadata() != null) {
Set<MetadataEntry> metadataEntrySet = createMetadata(request.getMetadata());
sampleService.mergeSampleMetadata(sample, metadataEntrySet);
}
return join.getObject().getId();
}

/**
* Update samples in a project
*
* @param requests Each {@link UpdateSampleRequest} contains details about the sample to update
* @param locale Users current locale
* @return result of creating the samples
* @throws UIMetadataImportException if there are errors creating the project samples
*/
public String updateSamples(UpdateSampleRequest[] requests, Locale locale) throws UIMetadataImportException {
Map<String, String> errors = new HashMap<>();
for (UpdateSampleRequest request : requests) {
try {
updateSample(request);
} catch (Exception e) {
errors.put(request.getName(), e.getMessage());
}
return ResponseEntity.ok(new AjaxCreateItemSuccessResponse(join.getObject().getId()));
} catch (EntityNotFoundException e) {
return ResponseEntity.ok(new AjaxErrorResponse(
messageSource.getMessage("server.AddSample.error.exists", new Object[] {}, locale)));
}
if (errors.isEmpty()) {
return messageSource.getMessage("server.AddSample.success", null, locale);
} else {
throw new UIMetadataImportException(errors);
}
}

/**
* Update a sample in a project
*
* @param request {@link UpdateSampleRequest} details about the sample to update
* @param sampleId Identifier for the sample
* @param locale Users current locale
* @param request {@link UpdateSampleRequest} contains details about the sample to update
* @return result of creating the sample
* @throws EntityNotFoundException if the identifier does not exist in the database
* @throws ConstraintViolationException if the entity being updated contains constraint violations
*/
@Transactional
public ResponseEntity<AjaxResponse> updateSample(UpdateSampleRequest request, Long sampleId, Locale locale) {
try {
Sample sample = sampleService.read(sampleId);
sample.setSampleName(request.getName());
if (!Strings.isNullOrEmpty(request.getOrganism())) {
sample.setOrganism(request.getOrganism());
}
if (request.getDescription() != null) {
sample.setDescription(request.getDescription());
}
if (request.getMetadata() != null) {
Set<MetadataEntry> metadataEntrySet = createMetadata(request.getMetadata());
sampleService.mergeSampleMetadata(sample, metadataEntrySet);
}
sampleService.update(sample);
return ResponseEntity.ok(new AjaxUpdateItemSuccessResponse(
messageSource.getMessage("server.AddSample.success", null, locale)));
} catch (Exception e) {
return ResponseEntity.status(HttpStatus.CONFLICT).body(new AjaxErrorResponse(e.getMessage()));
public Sample updateSample(UpdateSampleRequest request)
throws EntityNotFoundException, ConstraintViolationException {
Long sampleId = request.getSampleId();
Sample sample = sampleService.read(sampleId);
sample.setSampleName(request.getName());
if (!Strings.isNullOrEmpty(request.getOrganism())) {
sample.setOrganism(request.getOrganism());
}
if (request.getDescription() != null) {
sample.setDescription(request.getDescription());
}
if (request.getMetadata() != null) {
Set<MetadataEntry> metadataEntrySet = createMetadata(request.getMetadata());
sampleService.updateSampleMetadata(sample, metadataEntrySet);
}
return sampleService.update(sample);

}

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>

<databaseChangeLog xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd"></databaseChangeLog>
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd">
<include file="metadata-field-add-label-constraint.xml" relativeToChangelogFile="true" />
</databaseChangeLog>
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>

<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-4.3.xsd">
<changeSet id="metadata-field-add-label-constraint" author="katherine">
<addUniqueConstraint columnNames="label" tableName="metadata_field" constraintName="UK_METADATA_FIELD_LABEL" />
</changeSet>
</databaseChangeLog>
34 changes: 5 additions & 29 deletions src/main/webapp/resources/js/apis/projects/samples.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,29 +129,24 @@ export async function getLockedSamples({
return response.data;
joshsadam marked this conversation as resolved.
Show resolved Hide resolved
}

export async function createSample({
export async function createSamples({
projectId,
body,
}: {
projectId: string;
body: SampleRequest;
body: SampleRequest[];
}) {
return await axios.post(`${URL}/${projectId}/samples/add-sample`, body);
}

export async function updateSample({
export async function updateSamples({
projectId,
sampleId,
body,
}: {
projectId: string;
sampleId: number;
body: SampleRequest;
body: SampleRequest[];
}) {
return await axios.patch(
`${URL}/${projectId}/samples/add-sample/${sampleId}`,
body
);
return await axios.patch(`${URL}/${projectId}/samples/add-sample`, body);
}

/**
Expand All @@ -168,25 +163,6 @@ export async function validateSampleName(name: string) {
return response.json();
}

/**
* Create a new sample within a project
* @param name - name of the new sample
* @param organism - name of the organism (optional)
* @returns {Promise<Response>}
*/
export async function createNewSample({
name,
organism,
}: {
name: string;
organism: string;
}) {
return post(`${URL}/${PROJECT_ID}/samples/add-sample`, {
name: name.trim(),
organism,
});
}

/**
* Share or move samples with another project.
* @param currentId - current projectId
Expand Down
Loading