Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix recreate symlink issue + mac2unix/dos2unix replacement #106

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

fix recreate symlink issue + mac2unix/dos2unix replacement #106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
05d3bce
permanant fix when there is no mac2unix/dos2unix
RoanKanninga Sep 6, 2023
871db14
fix
RoanKanninga Mar 7, 2024
17edce9
Merge branch 'master' of https://github.com/molgenis/NGS_Demultiplexing
RoanKanninga Mar 11, 2024
30f7aa9
fixing logging (now in line with NGS_Automated)
RoanKanninga Mar 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion generate_template.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,11 @@ rm -rf "${WORKDIR}/generatedscripts/NGS_Demultiplexing/${RAWDATANAME}/out.csv"
#
sampsheet="${WORKDIR}/generatedscripts/NGS_Demultiplexing/${RAWDATANAME}/${RAWDATANAME}.csv"

mac2unix "${sampsheet}"
SAMPLESHEET_SEP=","
cp "${sampsheet}"{,.converted}
sed -i 's/\r/\n/g' "${sampsheet}.converted"
Copy link
Member

@pneerincx pneerincx Mar 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
sed -i 's/\r/\n/g' "${sampsheet}.converted"
printf '\n' >> "${sampsheet}.converted"
sed -i 's/\r/\n/g' "${sampsheet}.converted"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The updated code replaces both mac2unix as well as dos2unix resulting whatever2unix. Another typical error is a missing line end character on the last line. The suggestion above inserts a line end character before the
sed -i "/^[\s${SAMPLESHEET_SEP}]*$/d" "${sampsheet}.converted"
line which removes empty lines. So if it was not necessary to add another line end character the sed that removes empty lines will fix it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check, updated

sed -i "/^[\s${SAMPLESHEET_SEP}]*$/d" "${sampsheet}.converted"
mv "${sampsheet}.converted" "${sampsheet}"

declare -a _sampleSheetColumnNames=()
declare -A _sampleSheetColumnOffsets=()
Expand Down
8 changes: 4 additions & 4 deletions protocols/Demultiplex.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,8 +270,8 @@ discarded=$(fgrep "DISCARDED" ${runPrefix}.demultiplex.log | awk -F '[()]' '{pri
if [[ "${discarded}" -gt 75 ]]
then
echo "discarded percentage (${discarded}%) is higher than 75 procent, exiting"
echo -e "Dear Helpdesk ,\ndiscarded percentage (${discarded}%) is higher than 75 procent. See ${ngsDir}/${runPrefix}.demultiplex.log for a complete overview of all barcodes.\nThe demultiplexing pipeline has quit for sequence run ${runPrefix}, please check if the correct barcodes are enterd in the samplesheet\nkind regards GCC" >> "${logsDir}/${filePrefix}/${filePrefix}.demultiplexing.discarded"
cat ${runPrefix}.demultiplex.log >> "${logsDir}/${filePrefix}/${filePrefix}.demultiplexing.discarded"
echo -e "Dear Helpdesk ,\ndiscarded percentage (${discarded}%) is higher than 75 procent. See ${ngsDir}/${runPrefix}.demultiplex.log for a complete overview of all barcodes.\nThe demultiplexing pipeline has quit for sequence run ${runPrefix}, please check if the correct barcodes are enterd in the samplesheet\nkind regards GCC" >> "${logsDir}/${filePrefix}/run01.demultiplexing.discarded"
cat ${runPrefix}.demultiplex.log >> "${logsDir}/${filePrefix}/run01.demultiplexing.discarded"

if [[ -r ../../../logs/${SCRIPT_NAME}.mailinglist ]]
then
Expand Down Expand Up @@ -310,12 +310,12 @@ then
echo -e "\nAlle lanen voor deze barcode(s) worden niet door de pipeline gehaald.\n\ngroeten,\nGCC" >> mailText.txt
cat mailText.txt
cat mailText.txt | mail -s "er zijn samples afgekeurd van run: ${runPrefix}" "${mailingList}"
cat mailText.txt >> "${logsDir}/${filePrefix}/${filePrefix}.demultiplexing.discarded"
cat mailText.txt >> "${logsDir}/${filePrefix}/run01.demultiplexing.discarded"

fi
else
echo "number of discarded reads is ${discarded}"
cat ${runPrefix}.demultiplex.log >> "${logsDir}/${filePrefix}/${filePrefix}.demultiplexing.finished"
cat ${runPrefix}.demultiplex.log >> "${logsDir}/${filePrefix}/run01.demultiplexing.finished"
fi


Expand Down
14 changes: 7 additions & 7 deletions protocols/UploadSampleSheet.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ then
fi


if [ ! -f "${workDir}/logs/${filePrefix}/${filePrefix}.${SCRIPT_NAME}.finished" ]
if [ ! -f "${workDir}/logs/${filePrefix}/run01.${SCRIPT_NAME}.finished" ]
then
if curl -s -f -H "Content-Type: application/json" -X POST -d "{"username"="${USERNAME}", "password"="${PASSWORD}"}" https://${MOLGENISSERVER}/api/v1/login
then
Expand All @@ -81,7 +81,7 @@ then
echo "curl couldn't connect to host, skipped the uploading of the samplesheet to ${MOLGENISSERVER}" > "${ngsDir}/${filePrefix}.csv.uploadingFailed"

fi
touch "${workDir}/logs/${filePrefix}/${filePrefix}.${SCRIPT_NAME}.finished"
touch "${workDir}/logs/${filePrefix}/run01.${SCRIPT_NAME}.finished"
else
echo "samplesheet already uploaded to ${MOLGENISSERVER}"

Expand All @@ -95,21 +95,21 @@ then
rm "${ngsDir}/rejectedBarcodes.txt"
fi

if [ -f "${workDir}/logs/${filePrefix}/${filePrefix}.demultiplexing.started" ]
if [ -f "${workDir}/logs/${filePrefix}/run01.demultiplexing.started" ]
then
mv "${workDir}/logs/${filePrefix}/${filePrefix}.demultiplexing."{started,finished}
mv "${workDir}/logs/${filePrefix}/run01.demultiplexing."{started,finished}
else
touch "${workDir}/logs/${filePrefix}/${filePrefix}.demultiplexing.finished"
touch "${workDir}/logs/${filePrefix}/run01.demultiplexing.finished"
fi
cd "${runResultsDir}" || exit

while IFS='\n' read -r line; do resultsArray+=("$line"); done < <(find "${ngsDir}/"*.*)
for i in "${resultsArray[@]}"
do
ln -s "${i}"
ln -sf "${i}"
done

cd - || exit

echo "made symlinks from the rawdata/ngs folder to the results folder: ${runResultsDir}"
echo "finished: $(date +%FT%T%z)" >> "${workDir}/logs/${filePrefix}//${filePrefix}.demultiplexing.totalRuntime"
echo "finished: $(date +%FT%T%z)" >> "${workDir}/logs/${filePrefix}//run01.demultiplexing.totalRuntime"
2 changes: 0 additions & 2 deletions protocols/processInterop.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,6 @@ then
fi


touch "${workDir}/logs/${filePrefix}/${filePrefix}.demultiplexing.started"


#################Dit stuk later aanpassen, naar hoe de diagnostiek het wil. Dit werkt ook nog niet, kan niet met komma getallen overweg.##################
#if [ "${Q30}" -lt 100 ]
Expand Down
2 changes: 1 addition & 1 deletion templates/slurm/header_tnt.ftl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ export mydate_start
#
<#noparse>
runName=$(basename $(cd ../ && pwd ))
MC_failedFile="${logsDirectory}/${runName}.demultiplexing.failed"
MC_failedFile="${logsDirectory}/run01.demultiplexing.failed"


declare MC_singleSeperatorLine=$(head -c 120 /dev/zero | tr '\0' '-')
Expand Down