problems with particle extraction from 2d crystal #242
Comments
|
Dear Ethan,
The "No handlers could be found for logger "mrcz.test_mrcz" message is just
a warning, not an error. It can be safely ignored.
But this "then: then/endif not found." error is indeed something we have to
look into.
Some questions:
1) Did you update FOCUS in between the different runs of particle picking?
If yes, some bug could have been introduced.
2) How many "more images" did you include in your second run with 8
threads? And were you outputting the new picked particles into the same
directory as before (the default is SPR/) \, or a different one? If the
same, that could lead to some unexpected errors.
3) Are you splitting your particle stacks per-crystal or per-particle?
There's an option for that. If you are picking particles from a single 2D
crystal image you cannot split per-crystal, because you have only one.
Hope this helps and thanks for using FOCUS!
…--
Ricardo Diogo Righetto
Em sex, 20 de set de 2019 às 05:09, EthanJUCB <[email protected]>
escreveu:
Hi, I have some problems during particle extraction.
In the beginning, I was able to extract particles from several images with
the error" No handlers could be found for logger "mrcz.test_mrcz" . I
successfully had phase flipped stacks for crystal averages.
Then I tried to include more images using 8 threads. Somehow, only 1
thread was running according to system monitor and scripts stopped after
extracting particles from up to 4 images no matter how many image I
selected.
After that, I had the worst one, script completely stopped extracting even
for a single image. It now shows similar error"No handlers could be found
for logger "mrcz.test_mrcz" but with additional message " then: then/endif
not found."
I have no idea how to fix it now. Please see attached log. Many thanks.
Best
Xi
###########################PickExportParticleStack############################
############################################
Parameters requested by the Script
############################################
thread_number = 8
scriptname =
spr_sample_pixel = 0.5568
spr_doresample = n
spr_resmin = 100.0
spr_resmax = 10.0
spr_phaseshift = 0.0,180.0
spr_boxsize = 200
spr_sigcc = 0.3
spr_invertcontrast = y
spr_normalizebox = y
spr_calculatedefocustilted = y
spr_savephaseflipped = y
spr_savectfmultiplied = n
spr_savewienerfiltered = n
spr_wienerconstant = 0.4
spr_sigmanorm = 1.0
spr_radnorm = 0.90
spr_whichtiltgeometry = Defocus
spr_whichctf = Particle
spr_whichhalfset = particle-based
spr_savepickfig = y
cs = 2.6
kv = 300
sample_pixel = 0.5568
phacon = 0.995
magnification = 89798.9
use_masked_image = n
spr_ignore_lat2 = y
spr_ignore_c1 = y
spr_createstarfile = y
spr_dir = SPR
spr_imgs_dir =
spr_mergefile =
spr_stacks_dir =
spr_picking_dir =
spr_stack_rootname =
############################################
Parameters from the Preferences settings
############################################
app_python = /usr/bin/python2.7
app_cadaver = /usr/bin/cadaver
app_anaconda = /usr/bin/python2.7
dir_ccp4 = /usr/local/ccp4-7.0
app_gawk = /usr/bin/gawk
cuda_library = /usr/local/cuda/lib64
app_unblur =
/usr/local/cina/unblur/unblur_1.0.2/bin/unblur_openmp_7_17_15.exe
app_motioncor2 = /home/xijiang/Downloads/Motioncor2/MotionCor2-01-30-2017
app_gctf =
/home/xijiang/Downloads/Gctf_v1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64
app_ctffind = /home/xijiang/Downloads/CTFFIND4/bin/ctffind
app_gautomatch =
/home/xijiang/Downloads/Gautomatch_v0.53/bin/Gautomatch-v0.53_sm_20_cu8.0_x86_64
dir_eman2 = /usr/local/EMAN2/bin
dir_imod = /usr/local/IMOD
dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
dir_relion = /usr/local/relion3
Microscope_Name = CryoEM
Default_CS = 2.6
Default_KV = 300.0
Default_phase_shift_doit = n
status_folder_update = n
status_folder = none
status_microscope = Titan
status_refresh_rate = 0
status_webdav = http://my.server.com/files
website_blur_doit = y
website_blurkernel = 7
SCRATCH_DISK = .
FAST_DISK = .
GPU_how_many = 1
GPU_to_use = 0
GPU_cycle = y
MotionCor2_alone = y
notify_email_admin = y
notify_email_admin_adress = ***@***.***
notify_last_minutes = 15
notify_delay_minutes = 60
notify_lastnumber = 50
notify_limit_mild = 30
notify_limit_severe = 15
flag_isdark_threshold = 0.1
flag_hasstripe_threshold = 38.0
flag_hastoohighdefocus_threshold = 7.0
flag_hastoolowdefocus_threshold = 0.2
flag_icinesstoohigh_threshold = 1.6
flag_drifttoohigh_threshold = 50.0
flag_astigmatismtoohigh_threshold = 0.03
flag_ctfresolutiontoobad_threshold = 8.0
#############################################
Now the remainder of the original script:
#############################################
++++++++++++++++++++++++++++++++
Initializing script environment
++++++++++++++++++++++++++++++++
Running the following shell:
version tcsh 6.18.01 (Astron) 2012-02-14 (x86_64-unknown-linux) options
wide,nls,dl,al,kan,rh,color,filec
date = Thu Sep 19 20:01:59 PDT 2019
system = Linux
bin_2dx = /opt/focus/kernel/mrc/bin
proc_2dx = /opt/focus/scripts/proc
==== dir_eman2 = /usr/local/EMAN2/bin
=================================================================================================================================================================
==== app_python = /usr/bin/python2.7
==================================================================================================================================================================
==== app_anaconda = /usr/bin/python2.7
================================================================================================================================================================
==== dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
===================================================================================================================================
==== dir_relion = /usr/local/relion3
==================================================================================================================================================================
==== dir_ccp4 = /usr/local/ccp4-7.0
===================================================================================================================================================================
Finding CSH setup script from CCP4 installation..
Sourcing /usr/local/ccp4-7.0/bin/ccp4.setup-csh
Finished with /usr/local/ccp4-7.0/bin/ccp4.setup-csh
Will use 1 threads for optimal load balancing.
[1] 18154
No handlers could be found for logger "mrcz.test_mrcz"
Job 1/1 picking particles from micrographs 1 to 1...
[1] Done /usr/bin/python2.7
/opt/focus/scripts/proc/SPR_ExtractParticles.py ../
SPR/2dx_merge_dirfile-unique.dat SPR/picking/ SPR/stacks/ particles 200
0.0,180.0 0.5568 300 2.6 0.995 89798.9 0.3 y y y y n n 0.4 1.0 0.90 Defocus
Particle y n n 1 1
Now organizing and converting partial stack(s) to single .mrcs and .par
files...
then: then/endif not found.
::######################PickExportParticleStack
finished.#######################
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#242>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AB2W674SX2BBXPANDJKUTXDQKQ5G5ANCNFSM4IYSFZDA>
.
|
|
Hi Ricardo,
Many thanks.
I didn’t update FOCUS during the different runs. Then I uninstalled and installed again, it didn’t solve the problem.
I included 328 images but it stopped after 4 images. Yes, I used the default SPR folder. It looked like the current Par file will be overwrote or renamed with “old" if I started a new extraction. I tried to use different folder, It didn’t work either.
I split particles per crystal since I had two images in the beginning and it worked well.
Do you have any suggestions that how to solve this strange issue? Is it a good idea to install a fresh centos7 with only focus and its dependencies.
PS. I read the Nature Comm. paper about 2DX SPA processing. I wonder the 2D averaged images of crystals were obtained by averaging all boxes in a stack of one micrograph or they were obtained by using 2D classification method such like maximum likelihood in Relion.
Best
Xi
… On Sep 20, 2019, at 1:36 AM, Ricardo Righetto ***@***.***> wrote:
Dear Ethan,
The "No handlers could be found for logger "mrcz.test_mrcz" message is just
a warning, not an error. It can be safely ignored.
But this "then: then/endif not found." error is indeed something we have to
look into.
Some questions:
1) Did you update FOCUS in between the different runs of particle picking?
If yes, some bug could have been introduced.
2) How many "more images" did you include in your second run with 8
threads? And were you outputting the new picked particles into the same
directory as before (the default is SPR/) \, or a different one? If the
same, that could lead to some unexpected errors.
3) Are you splitting your particle stacks per-crystal or per-particle?
There's an option for that. If you are picking particles from a single 2D
crystal image you cannot split per-crystal, because you have only one.
Hope this helps and thanks for using FOCUS!
--
Ricardo Diogo Righetto
Em sex, 20 de set de 2019 às 05:09, EthanJUCB ***@***.***>
escreveu:
> Hi, I have some problems during particle extraction.
> In the beginning, I was able to extract particles from several images with
> the error" No handlers could be found for logger "mrcz.test_mrcz" . I
> successfully had phase flipped stacks for crystal averages.
>
> Then I tried to include more images using 8 threads. Somehow, only 1
> thread was running according to system monitor and scripts stopped after
> extracting particles from up to 4 images no matter how many image I
> selected.
>
> After that, I had the worst one, script completely stopped extracting even
> for a single image. It now shows similar error"No handlers could be found
> for logger "mrcz.test_mrcz" but with additional message " then: then/endif
> not found."
>
> I have no idea how to fix it now. Please see attached log. Many thanks.
>
> Best
> Xi
>
>
> ###########################PickExportParticleStack############################
>
> ############################################
> Parameters requested by the Script
>
> ############################################
> thread_number = 8
> scriptname =
> spr_sample_pixel = 0.5568
> spr_doresample = n
> spr_resmin = 100.0
> spr_resmax = 10.0
> spr_phaseshift = 0.0,180.0
> spr_boxsize = 200
> spr_sigcc = 0.3
> spr_invertcontrast = y
> spr_normalizebox = y
> spr_calculatedefocustilted = y
> spr_savephaseflipped = y
> spr_savectfmultiplied = n
> spr_savewienerfiltered = n
> spr_wienerconstant = 0.4
> spr_sigmanorm = 1.0
> spr_radnorm = 0.90
> spr_whichtiltgeometry = Defocus
> spr_whichctf = Particle
> spr_whichhalfset = particle-based
> spr_savepickfig = y
> cs = 2.6
> kv = 300
> sample_pixel = 0.5568
> phacon = 0.995
> magnification = 89798.9
> use_masked_image = n
> spr_ignore_lat2 = y
> spr_ignore_c1 = y
> spr_createstarfile = y
> spr_dir = SPR
> spr_imgs_dir =
> spr_mergefile =
> spr_stacks_dir =
> spr_picking_dir =
> spr_stack_rootname =
>
> ############################################
> Parameters from the Preferences settings
>
> ############################################
> app_python = /usr/bin/python2.7
> app_cadaver = /usr/bin/cadaver
> app_anaconda = /usr/bin/python2.7
> dir_ccp4 = /usr/local/ccp4-7.0
> app_gawk = /usr/bin/gawk
> cuda_library = /usr/local/cuda/lib64
> app_unblur =
> /usr/local/cina/unblur/unblur_1.0.2/bin/unblur_openmp_7_17_15.exe
> app_motioncor2 = /home/xijiang/Downloads/Motioncor2/MotionCor2-01-30-2017
> app_gctf =
> /home/xijiang/Downloads/Gctf_v1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64
> app_ctffind = /home/xijiang/Downloads/CTFFIND4/bin/ctffind
> app_gautomatch =
> /home/xijiang/Downloads/Gautomatch_v0.53/bin/Gautomatch-v0.53_sm_20_cu8.0_x86_64
> dir_eman2 = /usr/local/EMAN2/bin
> dir_imod = /usr/local/IMOD
> dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
> dir_relion = /usr/local/relion3
> Microscope_Name = CryoEM
> Default_CS = 2.6
> Default_KV = 300.0
> Default_phase_shift_doit = n
> status_folder_update = n
> status_folder = none
> status_microscope = Titan
> status_refresh_rate = 0
> status_webdav = http://my.server.com/files
> website_blur_doit = y
> website_blurkernel = 7
> SCRATCH_DISK = .
> FAST_DISK = .
> GPU_how_many = 1
> GPU_to_use = 0
> GPU_cycle = y
> MotionCor2_alone = y
> notify_email_admin = y
> notify_email_admin_adress = ***@***.***
> notify_last_minutes = 15
> notify_delay_minutes = 60
> notify_lastnumber = 50
> notify_limit_mild = 30
> notify_limit_severe = 15
> flag_isdark_threshold = 0.1
> flag_hasstripe_threshold = 38.0
> flag_hastoohighdefocus_threshold = 7.0
> flag_hastoolowdefocus_threshold = 0.2
> flag_icinesstoohigh_threshold = 1.6
> flag_drifttoohigh_threshold = 50.0
> flag_astigmatismtoohigh_threshold = 0.03
> flag_ctfresolutiontoobad_threshold = 8.0
>
> #############################################
> Now the remainder of the original script:
>
> #############################################
>
> ++++++++++++++++++++++++++++++++
> Initializing script environment
> ++++++++++++++++++++++++++++++++
>
> Running the following shell:
> version tcsh 6.18.01 (Astron) 2012-02-14 (x86_64-unknown-linux) options
> wide,nls,dl,al,kan,rh,color,filec
>
> date = Thu Sep 19 20:01:59 PDT 2019
> system = Linux
> bin_2dx = /opt/focus/kernel/mrc/bin
> proc_2dx = /opt/focus/scripts/proc
> ==== dir_eman2 = /usr/local/EMAN2/bin
> =================================================================================================================================================================
> ==== app_python = /usr/bin/python2.7
> ==================================================================================================================================================================
> ==== app_anaconda = /usr/bin/python2.7
> ================================================================================================================================================================
> ==== dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
> ===================================================================================================================================
> ==== dir_relion = /usr/local/relion3
> ==================================================================================================================================================================
> ==== dir_ccp4 = /usr/local/ccp4-7.0
> ===================================================================================================================================================================
> Finding CSH setup script from CCP4 installation..
> Sourcing /usr/local/ccp4-7.0/bin/ccp4.setup-csh
> Finished with /usr/local/ccp4-7.0/bin/ccp4.setup-csh
> Will use 1 threads for optimal load balancing.
> [1] 18154
> No handlers could be found for logger "mrcz.test_mrcz"
>
> Job 1/1 picking particles from micrographs 1 to 1...
>
> [1] Done /usr/bin/python2.7
> /opt/focus/scripts/proc/SPR_ExtractParticles.py ../
> SPR/2dx_merge_dirfile-unique.dat SPR/picking/ SPR/stacks/ particles 200
> 0.0,180.0 0.5568 300 2.6 0.995 89798.9 0.3 y y y y n n 0.4 1.0 0.90 Defocus
> Particle y n n 1 1
> Now organizing and converting partial stack(s) to single .mrcs and .par
> files...
> then: then/endif not found.
> ::######################PickExportParticleStack
> finished.#######################
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <#242>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AB2W674SX2BBXPANDJKUTXDQKQ5G5ANCNFSM4IYSFZDA>
> .
>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#242>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEVLNHBEFGFXBUSFI7EL7UTQKSDQTANCNFSM4IYSFZDA>.
|
|
Hi Richardo,
I tested extraction again on a fresh installed computer. Extraction worked for 1 image perfectly, once I included more images it stopped working and show the same error message as I posed.
I looked into and run the extraction script in terminal, it seemed like python was not able to import ioMrc, focus_utility and focus_ctf properly. I have no idea whey it didn’t import them properly as long as it worked in the beginning for 1 but not for many images later.
Best
Xi
… On Sep 20, 2019, at 1:36 AM, Ricardo Righetto ***@***.***> wrote:
Dear Ethan,
The "No handlers could be found for logger "mrcz.test_mrcz" message is just
a warning, not an error. It can be safely ignored.
But this "then: then/endif not found." error is indeed something we have to
look into.
Some questions:
1) Did you update FOCUS in between the different runs of particle picking?
If yes, some bug could have been introduced.
2) How many "more images" did you include in your second run with 8
threads? And were you outputting the new picked particles into the same
directory as before (the default is SPR/) \, or a different one? If the
same, that could lead to some unexpected errors.
3) Are you splitting your particle stacks per-crystal or per-particle?
There's an option for that. If you are picking particles from a single 2D
crystal image you cannot split per-crystal, because you have only one.
Hope this helps and thanks for using FOCUS!
--
Ricardo Diogo Righetto
Em sex, 20 de set de 2019 às 05:09, EthanJUCB ***@***.***>
escreveu:
> Hi, I have some problems during particle extraction.
> In the beginning, I was able to extract particles from several images with
> the error" No handlers could be found for logger "mrcz.test_mrcz" . I
> successfully had phase flipped stacks for crystal averages.
>
> Then I tried to include more images using 8 threads. Somehow, only 1
> thread was running according to system monitor and scripts stopped after
> extracting particles from up to 4 images no matter how many image I
> selected.
>
> After that, I had the worst one, script completely stopped extracting even
> for a single image. It now shows similar error"No handlers could be found
> for logger "mrcz.test_mrcz" but with additional message " then: then/endif
> not found."
>
> I have no idea how to fix it now. Please see attached log. Many thanks.
>
> Best
> Xi
>
>
> ###########################PickExportParticleStack############################
>
> ############################################
> Parameters requested by the Script
>
> ############################################
> thread_number = 8
> scriptname =
> spr_sample_pixel = 0.5568
> spr_doresample = n
> spr_resmin = 100.0
> spr_resmax = 10.0
> spr_phaseshift = 0.0,180.0
> spr_boxsize = 200
> spr_sigcc = 0.3
> spr_invertcontrast = y
> spr_normalizebox = y
> spr_calculatedefocustilted = y
> spr_savephaseflipped = y
> spr_savectfmultiplied = n
> spr_savewienerfiltered = n
> spr_wienerconstant = 0.4
> spr_sigmanorm = 1.0
> spr_radnorm = 0.90
> spr_whichtiltgeometry = Defocus
> spr_whichctf = Particle
> spr_whichhalfset = particle-based
> spr_savepickfig = y
> cs = 2.6
> kv = 300
> sample_pixel = 0.5568
> phacon = 0.995
> magnification = 89798.9
> use_masked_image = n
> spr_ignore_lat2 = y
> spr_ignore_c1 = y
> spr_createstarfile = y
> spr_dir = SPR
> spr_imgs_dir =
> spr_mergefile =
> spr_stacks_dir =
> spr_picking_dir =
> spr_stack_rootname =
>
> ############################################
> Parameters from the Preferences settings
>
> ############################################
> app_python = /usr/bin/python2.7
> app_cadaver = /usr/bin/cadaver
> app_anaconda = /usr/bin/python2.7
> dir_ccp4 = /usr/local/ccp4-7.0
> app_gawk = /usr/bin/gawk
> cuda_library = /usr/local/cuda/lib64
> app_unblur =
> /usr/local/cina/unblur/unblur_1.0.2/bin/unblur_openmp_7_17_15.exe
> app_motioncor2 = /home/xijiang/Downloads/Motioncor2/MotionCor2-01-30-2017
> app_gctf =
> /home/xijiang/Downloads/Gctf_v1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64
> app_ctffind = /home/xijiang/Downloads/CTFFIND4/bin/ctffind
> app_gautomatch =
> /home/xijiang/Downloads/Gautomatch_v0.53/bin/Gautomatch-v0.53_sm_20_cu8.0_x86_64
> dir_eman2 = /usr/local/EMAN2/bin
> dir_imod = /usr/local/IMOD
> dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
> dir_relion = /usr/local/relion3
> Microscope_Name = CryoEM
> Default_CS = 2.6
> Default_KV = 300.0
> Default_phase_shift_doit = n
> status_folder_update = n
> status_folder = none
> status_microscope = Titan
> status_refresh_rate = 0
> status_webdav = http://my.server.com/files
> website_blur_doit = y
> website_blurkernel = 7
> SCRATCH_DISK = .
> FAST_DISK = .
> GPU_how_many = 1
> GPU_to_use = 0
> GPU_cycle = y
> MotionCor2_alone = y
> notify_email_admin = y
> notify_email_admin_adress = ***@***.***
> notify_last_minutes = 15
> notify_delay_minutes = 60
> notify_lastnumber = 50
> notify_limit_mild = 30
> notify_limit_severe = 15
> flag_isdark_threshold = 0.1
> flag_hasstripe_threshold = 38.0
> flag_hastoohighdefocus_threshold = 7.0
> flag_hastoolowdefocus_threshold = 0.2
> flag_icinesstoohigh_threshold = 1.6
> flag_drifttoohigh_threshold = 50.0
> flag_astigmatismtoohigh_threshold = 0.03
> flag_ctfresolutiontoobad_threshold = 8.0
>
> #############################################
> Now the remainder of the original script:
>
> #############################################
>
> ++++++++++++++++++++++++++++++++
> Initializing script environment
> ++++++++++++++++++++++++++++++++
>
> Running the following shell:
> version tcsh 6.18.01 (Astron) 2012-02-14 (x86_64-unknown-linux) options
> wide,nls,dl,al,kan,rh,color,filec
>
> date = Thu Sep 19 20:01:59 PDT 2019
> system = Linux
> bin_2dx = /opt/focus/kernel/mrc/bin
> proc_2dx = /opt/focus/scripts/proc
> ==== dir_eman2 = /usr/local/EMAN2/bin
> =================================================================================================================================================================
> ==== app_python = /usr/bin/python2.7
> ==================================================================================================================================================================
> ==== app_anaconda = /usr/bin/python2.7
> ================================================================================================================================================================
> ==== dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
> ===================================================================================================================================
> ==== dir_relion = /usr/local/relion3
> ==================================================================================================================================================================
> ==== dir_ccp4 = /usr/local/ccp4-7.0
> ===================================================================================================================================================================
> Finding CSH setup script from CCP4 installation..
> Sourcing /usr/local/ccp4-7.0/bin/ccp4.setup-csh
> Finished with /usr/local/ccp4-7.0/bin/ccp4.setup-csh
> Will use 1 threads for optimal load balancing.
> [1] 18154
> No handlers could be found for logger "mrcz.test_mrcz"
>
> Job 1/1 picking particles from micrographs 1 to 1...
>
> [1] Done /usr/bin/python2.7
> /opt/focus/scripts/proc/SPR_ExtractParticles.py ../
> SPR/2dx_merge_dirfile-unique.dat SPR/picking/ SPR/stacks/ particles 200
> 0.0,180.0 0.5568 300 2.6 0.995 89798.9 0.3 y y y y n n 0.4 1.0 0.90 Defocus
> Particle y n n 1 1
> Now organizing and converting partial stack(s) to single .mrcs and .par
> files...
> then: then/endif not found.
> ::######################PickExportParticleStack
> finished.#######################
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <#242>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AB2W674SX2BBXPANDJKUTXDQKQ5G5ANCNFSM4IYSFZDA>
> .
>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#242>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEVLNHBEFGFXBUSFI7EL7UTQKSDQTANCNFSM4IYSFZDA>.
|
|
OK, so it seems there is a bug in the script if you try to run it more than
once. I will look into that and get back to you soon!
Regarding your question about the 2D crystal averages, they are simply the
average of all boxes extracted from a given crystal. No 2D classification
or alignment were performed. This is the next script in the FOCUS pipeline
after the "Pick & Extract Particles".
Best wishes,
…--
Ricardo Diogo Righetto
Em dom, 22 de set de 2019 às 01:15, EthanJUCB <[email protected]>
escreveu:
Hi Richardo,
I tested extraction again on a fresh installed computer. Extraction worked
for 1 image perfectly, once I included more images it stopped working and
show the same error message as I posed.
I looked into and run the extraction script in terminal, it seemed like
python was not able to import ioMrc, focus_utility and focus_ctf properly.
I have no idea whey it didn’t import them properly as long as it worked in
the beginning for 1 but not for many images later.
Best
Xi
> On Sep 20, 2019, at 1:36 AM, Ricardo Righetto ***@***.***>
wrote:
>
> Dear Ethan,
>
> The "No handlers could be found for logger "mrcz.test_mrcz" message is
just
> a warning, not an error. It can be safely ignored.
> But this "then: then/endif not found." error is indeed something we have
to
> look into.
> Some questions:
>
> 1) Did you update FOCUS in between the different runs of particle
picking?
> If yes, some bug could have been introduced.
> 2) How many "more images" did you include in your second run with 8
> threads? And were you outputting the new picked particles into the same
> directory as before (the default is SPR/) \, or a different one? If the
> same, that could lead to some unexpected errors.
> 3) Are you splitting your particle stacks per-crystal or per-particle?
> There's an option for that. If you are picking particles from a single 2D
> crystal image you cannot split per-crystal, because you have only one.
>
> Hope this helps and thanks for using FOCUS!
>
>
>
>
> --
> Ricardo Diogo Righetto
>
>
> Em sex, 20 de set de 2019 às 05:09, EthanJUCB ***@***.***>
> escreveu:
>
> > Hi, I have some problems during particle extraction.
> > In the beginning, I was able to extract particles from several images
with
> > the error" No handlers could be found for logger "mrcz.test_mrcz" . I
> > successfully had phase flipped stacks for crystal averages.
> >
> > Then I tried to include more images using 8 threads. Somehow, only 1
> > thread was running according to system monitor and scripts stopped
after
> > extracting particles from up to 4 images no matter how many image I
> > selected.
> >
> > After that, I had the worst one, script completely stopped extracting
even
> > for a single image. It now shows similar error"No handlers could be
found
> > for logger "mrcz.test_mrcz" but with additional message " then:
then/endif
> > not found."
> >
> > I have no idea how to fix it now. Please see attached log. Many thanks.
> >
> > Best
> > Xi
> >
> >
> >
###########################PickExportParticleStack############################
> >
> > ############################################
> > Parameters requested by the Script
> >
> > ############################################
> > thread_number = 8
> > scriptname =
> > spr_sample_pixel = 0.5568
> > spr_doresample = n
> > spr_resmin = 100.0
> > spr_resmax = 10.0
> > spr_phaseshift = 0.0,180.0
> > spr_boxsize = 200
> > spr_sigcc = 0.3
> > spr_invertcontrast = y
> > spr_normalizebox = y
> > spr_calculatedefocustilted = y
> > spr_savephaseflipped = y
> > spr_savectfmultiplied = n
> > spr_savewienerfiltered = n
> > spr_wienerconstant = 0.4
> > spr_sigmanorm = 1.0
> > spr_radnorm = 0.90
> > spr_whichtiltgeometry = Defocus
> > spr_whichctf = Particle
> > spr_whichhalfset = particle-based
> > spr_savepickfig = y
> > cs = 2.6
> > kv = 300
> > sample_pixel = 0.5568
> > phacon = 0.995
> > magnification = 89798.9
> > use_masked_image = n
> > spr_ignore_lat2 = y
> > spr_ignore_c1 = y
> > spr_createstarfile = y
> > spr_dir = SPR
> > spr_imgs_dir =
> > spr_mergefile =
> > spr_stacks_dir =
> > spr_picking_dir =
> > spr_stack_rootname =
> >
> > ############################################
> > Parameters from the Preferences settings
> >
> > ############################################
> > app_python = /usr/bin/python2.7
> > app_cadaver = /usr/bin/cadaver
> > app_anaconda = /usr/bin/python2.7
> > dir_ccp4 = /usr/local/ccp4-7.0
> > app_gawk = /usr/bin/gawk
> > cuda_library = /usr/local/cuda/lib64
> > app_unblur =
> > /usr/local/cina/unblur/unblur_1.0.2/bin/unblur_openmp_7_17_15.exe
> > app_motioncor2 =
/home/xijiang/Downloads/Motioncor2/MotionCor2-01-30-2017
> > app_gctf =
> > /home/xijiang/Downloads/Gctf_v1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64
> > app_ctffind = /home/xijiang/Downloads/CTFFIND4/bin/ctffind
> > app_gautomatch =
> >
/home/xijiang/Downloads/Gautomatch_v0.53/bin/Gautomatch-v0.53_sm_20_cu8.0_x86_64
> > dir_eman2 = /usr/local/EMAN2/bin
> > dir_imod = /usr/local/IMOD
> > dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
> > dir_relion = /usr/local/relion3
> > Microscope_Name = CryoEM
> > Default_CS = 2.6
> > Default_KV = 300.0
> > Default_phase_shift_doit = n
> > status_folder_update = n
> > status_folder = none
> > status_microscope = Titan
> > status_refresh_rate = 0
> > status_webdav = http://my.server.com/files
> > website_blur_doit = y
> > website_blurkernel = 7
> > SCRATCH_DISK = .
> > FAST_DISK = .
> > GPU_how_many = 1
> > GPU_to_use = 0
> > GPU_cycle = y
> > MotionCor2_alone = y
> > notify_email_admin = y
> > notify_email_admin_adress = ***@***.***
> > notify_last_minutes = 15
> > notify_delay_minutes = 60
> > notify_lastnumber = 50
> > notify_limit_mild = 30
> > notify_limit_severe = 15
> > flag_isdark_threshold = 0.1
> > flag_hasstripe_threshold = 38.0
> > flag_hastoohighdefocus_threshold = 7.0
> > flag_hastoolowdefocus_threshold = 0.2
> > flag_icinesstoohigh_threshold = 1.6
> > flag_drifttoohigh_threshold = 50.0
> > flag_astigmatismtoohigh_threshold = 0.03
> > flag_ctfresolutiontoobad_threshold = 8.0
> >
> > #############################################
> > Now the remainder of the original script:
> >
> > #############################################
> >
> > ++++++++++++++++++++++++++++++++
> > Initializing script environment
> > ++++++++++++++++++++++++++++++++
> >
> > Running the following shell:
> > version tcsh 6.18.01 (Astron) 2012-02-14 (x86_64-unknown-linux) options
> > wide,nls,dl,al,kan,rh,color,filec
> >
> > date = Thu Sep 19 20:01:59 PDT 2019
> > system = Linux
> > bin_2dx = /opt/focus/kernel/mrc/bin
> > proc_2dx = /opt/focus/scripts/proc
> > ==== dir_eman2 = /usr/local/EMAN2/bin
> >
=================================================================================================================================================================
> > ==== app_python = /usr/bin/python2.7
> >
==================================================================================================================================================================
> > ==== app_anaconda = /usr/bin/python2.7
> >
================================================================================================================================================================
> > ==== dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
> >
===================================================================================================================================
> > ==== dir_relion = /usr/local/relion3
> >
==================================================================================================================================================================
> > ==== dir_ccp4 = /usr/local/ccp4-7.0
> >
===================================================================================================================================================================
> > Finding CSH setup script from CCP4 installation..
> > Sourcing /usr/local/ccp4-7.0/bin/ccp4.setup-csh
> > Finished with /usr/local/ccp4-7.0/bin/ccp4.setup-csh
> > Will use 1 threads for optimal load balancing.
> > [1] 18154
> > No handlers could be found for logger "mrcz.test_mrcz"
> >
> > Job 1/1 picking particles from micrographs 1 to 1...
> >
> > [1] Done /usr/bin/python2.7
> > /opt/focus/scripts/proc/SPR_ExtractParticles.py ../
> > SPR/2dx_merge_dirfile-unique.dat SPR/picking/ SPR/stacks/ particles 200
> > 0.0,180.0 0.5568 300 2.6 0.995 89798.9 0.3 y y y y n n 0.4 1.0 0.90
Defocus
> > Particle y n n 1 1
> > Now organizing and converting partial stack(s) to single .mrcs and .par
> > files...
> > then: then/endif not found.
> > ::######################PickExportParticleStack
> > finished.#######################
> >
> > —
> > You are receiving this because you are subscribed to this thread.
> > Reply to this email directly, view it on GitHub
> > <
#242
>,
> > or mute the thread
> > <
https://github.com/notifications/unsubscribe-auth/AB2W674SX2BBXPANDJKUTXDQKQ5G5ANCNFSM4IYSFZDA
>
> > .
> >
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub <
#242>,
or mute the thread <
https://github.com/notifications/unsubscribe-auth/AEVLNHBEFGFXBUSFI7EL7UTQKSDQTANCNFSM4IYSFZDA
>.
>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#242>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AB2W674WPT7CS2IMHJPPKXLQK2TJVANCNFSM4IYSFZDA>
.
|
|
Dear Xi, I have found a bug in the splitting of particles into per-crystal half-sets, that yielded the following error message:
Have you ever encountered that? This bugfix is now available in the latest version of the source code and should be in the snapshot binaries from our website within the hour (https://focus.c-cina.unibas.ch/download.php). Please make sure you are using this latest version of the source code, either from Github or the daily binary snapshots. Other than that, I was not able to reproduce the issue. Extracting particles gives no error no matter how many images are included or how many times I run it, even on the same target directory for storing the stacks (e.g. SPR/). |
|
Hi Ricardo,
No, I never had this issue.
Seems like the problem is related to the libraries in my computer rather than bugs in 2dx.
I wonder which OS you use for testing and developing.
Best
Xi
… On Oct 8, 2019, at 9:22 AM, Ricardo Righetto ***@***.***> wrote:
Dear Xi,
I have found a bug in the splitting of particles into per-crystal half-sets, that yielded the following error message:
sed: invalid usage of line address 0
Have you ever encountered that? This bugfix is now available in the latest version of the source code and should be in the snapshot binaries from our website within the hour (https://focus.c-cina.unibas.ch/download.php <https://focus.c-cina.unibas.ch/download.php>). Please make sure you are using this latest version of the source code, either from Github or the daily binary snapshots.
Other than that, I was not able to reproduce the issue. Extracting particles gives no error no matter how many images are included or how many times I run it, even on the same target directory for storing the stacks (e.g. SPR/).
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#242>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEVLNHBZDDUPAAWPKCCI5M3QNSXUPANCNFSM4IYSFZDA>.
|
|
Dear Xi, I use Ubuntu 18.04.3 LTS for running and developing FOCUS. But I used CentOS 7 until recently so it should work. Best wishes, |
|
I am having a related issue... I've processed drift-corrected images as 2d crystals and now I would like to pick particles from one of the images. I selected the image in the library and then attempted to run "Pick & export particles", and it appears to work but I don't get a window at the right of the GUI like I do with the 2d crystal scripts, so I think something is wrong... I also get some errors (see output below). I would appreciate any help/guidance- ###########################PickExportParticleStack############################ ############################################ Parameters requested by the Script############################################ ############################################ Parameters from the Preferences settings############################################ ############################################# Now the remainder of the original script:############################################# ++++++++++++++++++++++++++++++++ Job 1/1 picking particles from micrographs 1 to 1... Now boxing unit cells of micrograph 1/1. ../auto/0000000019/mimage_2dx.mrc |
|
Dear Amanda, You are getting two errors:
Thanks for using FOCUS, |
|
Hi Ricardo, (focusenv) [drennan@ad.wisc.edu@BIOCWK-01037L data]$ cat /mnt/data/top100tilt0/merge/LOGS/PickExportParticleStack.log ############################################ Parameters requested by the Script############################################ ############################################ Parameters from the Preferences settings############################################ ############################################# Now the remainder of the original script:############################################# :++++++++++++++++++++++++++++++++ :: ../auto/0000000019/mimage_2dx.mrc |
|
Dear Amanda, Thanks for sending the logfile. Does it work if you use a Python 3.X version? Make sure that Best wishes, |
|
Hi Ricardo, Successfully installed mrcz-0.5.4 numpy-1.18.1 ############################################ Parameters requested by the Script############################################ ############################################ Parameters from the Preferences settings############################################ ############################################# Now the remainder of the original script:############################################# :++++++++++++++++++++++++++++++++ |
|
OK, from this new error message I can confirm that somehow you are not running the latest version of the source code: This line, as written above, is not compatible with Python3, however the code has been updated a few months ago to be Python3 compliant... If you are updating from the source code ( Best wishes, |
|
Ok, thank you so much!!! |
|
Pick & export particles works after a fresh installation. |
|
Hi Focus team, I am experiencing the same error that Xi described above. I was wondering if a solution was found? Thanks! ::###########################PickExportParticleStack############################ ############################################ Parameters requested by the Script############################################ ############################################ Parameters from the Preferences settings############################################ ############################################# Now the remainder of the original script:############################################# :++++++++++++++++++++++++++++++++ Job 1/1 picking particles from micrographs 1 to 1... [1] Done /usr/bin/python3 /opt/sbgrid/x86_64-linux/focus/20201022/scripts/proc/SPR_ExtractParticles.py ../ SPR/2dx_merge_dirfile-unique.dat SPR/picking/ SPR/stacks/ particles 256 0.0,0.0 1.078 300 2.7 0.995 46382.2 0.0 y y y n y n 0.4 1.0 0.90 Defocus Particle y n y 1 1 |
|
Hi Bill, Apologies for not replying to your earlier requests! This looks like there was some error when extracting the particles.
Thanks! |
|
Hi Ricardo, Thank you for getting back to me. A "particles_1_r1.par" is being generated in the <project_path> /SPR/stacks folder, but the par file is empty. However, I have now been able extraction to work! I reinstalled all the needed dependencies, which might have done the trick (numexpr, pillow, matplotlib). Best, |
Hi, I have some problems during particle extraction.
In the beginning, I was able to extract particles from several images with the error" No handlers could be found for logger "mrcz.test_mrcz" . I successfully had phase flipped stacks for crystal averages.
Then I tried to include more images using 8 threads. Somehow, only 1 thread was running according to system monitor and scripts stopped after extracting particles from up to 4 images no matter how many image I selected.
After that, I had the worst one, script completely stopped extracting even for a single image. It now shows similar error"No handlers could be found for logger "mrcz.test_mrcz" but with additional message " then: then/endif not found."
I have no idea how to fix it now. Please see attached log. Many thanks.
Best
Xi
###########################PickExportParticleStack############################
############################################
Parameters requested by the Script
############################################
thread_number = 8
scriptname =
spr_sample_pixel = 0.5568
spr_doresample = n
spr_resmin = 100.0
spr_resmax = 10.0
spr_phaseshift = 0.0,180.0
spr_boxsize = 200
spr_sigcc = 0.3
spr_invertcontrast = y
spr_normalizebox = y
spr_calculatedefocustilted = y
spr_savephaseflipped = y
spr_savectfmultiplied = n
spr_savewienerfiltered = n
spr_wienerconstant = 0.4
spr_sigmanorm = 1.0
spr_radnorm = 0.90
spr_whichtiltgeometry = Defocus
spr_whichctf = Particle
spr_whichhalfset = particle-based
spr_savepickfig = y
cs = 2.6
kv = 300
sample_pixel = 0.5568
phacon = 0.995
magnification = 89798.9
use_masked_image = n
spr_ignore_lat2 = y
spr_ignore_c1 = y
spr_createstarfile = y
spr_dir = SPR
spr_imgs_dir =
spr_mergefile =
spr_stacks_dir =
spr_picking_dir =
spr_stack_rootname =
############################################
Parameters from the Preferences settings
############################################
app_python = /usr/bin/python2.7
app_cadaver = /usr/bin/cadaver
app_anaconda = /usr/bin/python2.7
dir_ccp4 = /usr/local/ccp4-7.0
app_gawk = /usr/bin/gawk
cuda_library = /usr/local/cuda/lib64
app_unblur = /usr/local/cina/unblur/unblur_1.0.2/bin/unblur_openmp_7_17_15.exe
app_motioncor2 = /home/xijiang/Downloads/Motioncor2/MotionCor2-01-30-2017
app_gctf = /home/xijiang/Downloads/Gctf_v1.06/bin/Gctf-v1.06_sm_30_cu8.0_x86_64
app_ctffind = /home/xijiang/Downloads/CTFFIND4/bin/ctffind
app_gautomatch = /home/xijiang/Downloads/Gautomatch_v0.53/bin/Gautomatch-v0.53_sm_20_cu8.0_x86_64
dir_eman2 = /usr/local/EMAN2/bin
dir_imod = /usr/local/IMOD
dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin
dir_relion = /usr/local/relion3
Microscope_Name = CryoEM
Default_CS = 2.6
Default_KV = 300.0
Default_phase_shift_doit = n
status_folder_update = n
status_folder = none
status_microscope = Titan
status_refresh_rate = 0
status_webdav = http://my.server.com/files
website_blur_doit = y
website_blurkernel = 7
SCRATCH_DISK = .
FAST_DISK = .
GPU_how_many = 1
GPU_to_use = 0
GPU_cycle = y
MotionCor2_alone = y
notify_email_admin = y
notify_email_admin_adress = [email protected]
notify_last_minutes = 15
notify_delay_minutes = 60
notify_lastnumber = 50
notify_limit_mild = 30
notify_limit_severe = 15
flag_isdark_threshold = 0.1
flag_hasstripe_threshold = 38.0
flag_hastoohighdefocus_threshold = 7.0
flag_hastoolowdefocus_threshold = 0.2
flag_icinesstoohigh_threshold = 1.6
flag_drifttoohigh_threshold = 50.0
flag_astigmatismtoohigh_threshold = 0.03
flag_ctfresolutiontoobad_threshold = 8.0
#############################################
Now the remainder of the original script:
#############################################
++++++++++++++++++++++++++++++++
Initializing script environment
++++++++++++++++++++++++++++++++
Running the following shell:
version tcsh 6.18.01 (Astron) 2012-02-14 (x86_64-unknown-linux) options wide,nls,dl,al,kan,rh,color,filec
date = Thu Sep 19 20:01:59 PDT 2019
system = Linux
bin_2dx = /opt/focus/kernel/mrc/bin
proc_2dx = /opt/focus/scripts/proc
==== dir_eman2 = /usr/local/EMAN2/bin =================================================================================================================================================================
==== app_python = /usr/bin/python2.7 ==================================================================================================================================================================
==== app_anaconda = /usr/bin/python2.7 ================================================================================================================================================================
==== dir_frealign = /home/xijiang/Downloads/frealign-2dx-master/bin ===================================================================================================================================
==== dir_relion = /usr/local/relion3 ==================================================================================================================================================================
==== dir_ccp4 = /usr/local/ccp4-7.0 ===================================================================================================================================================================
Finding CSH setup script from CCP4 installation..
Sourcing /usr/local/ccp4-7.0/bin/ccp4.setup-csh
Finished with /usr/local/ccp4-7.0/bin/ccp4.setup-csh
Will use 1 threads for optimal load balancing.
[1] 18154
No handlers could be found for logger "mrcz.test_mrcz"
Job 1/1 picking particles from micrographs 1 to 1...
[1] Done /usr/bin/python2.7 /opt/focus/scripts/proc/SPR_ExtractParticles.py ../ SPR/2dx_merge_dirfile-unique.dat SPR/picking/ SPR/stacks/ particles 200 0.0,180.0 0.5568 300 2.6 0.995 89798.9 0.3 y y y y n n 0.4 1.0 0.90 Defocus Particle y n n 1 1
Now organizing and converting partial stack(s) to single .mrcs and .par files...
then: then/endif not found.
::######################PickExportParticleStack finished.#######################
The text was updated successfully, but these errors were encountered: