Generating Numbers of Images with Stored Orientations

[david-yw-t]

I’m looking to generate large number (for example 10,000 images) of a PDB file at random orientations and also store the orientations for each image. Could you advise on the most efficient way to achieve this using Parakeet?

[jmp1985]

Hi @david-yw-t

Sure, if you want to do this you could use something like this:

$ cat config.yaml
microscope:
  beam:
    electrons_per_angstrom: 140
    energy: 300
  detector:
    nx: 1000
    ny: 1000
    pixel_size: 1
  lens:
    c_10: -20000
sample:
  box:
  - 1000
  - 1000
  - 1000
  centre:
  - 500
  - 500
  - 500
  molecules: 
    pdb:
      - id: 4v1w
  shape:
    cuboid:
      length_x: 1000.0
      length_y: 1000.0
      length_z: 200.0
    type: cuboid
scan:
  mode: single_particle 
  num_images: 10

This will generate 10 images of an apoferritin particle with random orientations. You can just set num_images to 10000 and that will generate more random orientations. You can run this with the following commands:

parakeet.sample.new -c config.yaml
parakeet.sample.add_molecules -c config.yaml
parakeet.simulate.exit_wave -c config.yaml
parakeet.simulate.optics -c config.yaml
parakeet.simulate.image -c config.yaml
parakeet.export image.h5 -o image.mrc

I hope this helps! Let me know if you have any issues!

[david-yw-t]

Thanks for your fast reply! It has been very helpful to me.

So what I gather, this creates 10 images with random tilts?

Opening file: image.h5
    Running job: 1/10 for image 0 fraction 0 with tilt -139.46818992367992 degrees
Simulating image 1
    Image min/max/mean: 87/202/1.4e+02
    Running job: 2/10 for image 1 fraction 0 with tilt -131.3488098531687 degrees
Simulating image 2
    Image min/max/mean: 84/200/1.4e+02
    Running job: 3/10 for image 2 fraction 0 with tilt 149.51828838271743 degrees

How do I get the orientations of the molecule? Am I correct that these are in axis-angle format in image.h5 in entry/data/tilt_alpha and tilt_axis_x, tilt_axis_y, tilt_axis_z?

Again thanks for your help!

[jmp1985]

Hi!

Sorry, I forgot to add that you can get the particle positions and orientations in relion star file format by doing:

parakeet.metadata.export -c config.yaml -s sample.h5

This should produce a directory called relion which will have a load of starfiles.

[david-yw-t]

Hi, thanks for your help. Much appreciated.

I tried that command on the v0.5.1 singularity container, and it throws an error:

Singularity> parakeet.metadata.export -c config.yaml -s sample.h5  
Traceback (most recent call last):
  File "/usr/local/bin/parakeet.metadata.export", line 8, in <module>
    sys.exit(export())
  File "/usr/local/lib/python3.8/dist-packages/parakeet/command_line/metadata/_export.py", line 100, in export
    export_impl(get_parser().parse_args(args=args))
  File "/usr/local/lib/python3.8/dist-packages/parakeet/command_line/metadata/_export.py", line 92, in export_impl
    parakeet.metadata.export(args.config, args.sample, args.directory, args.relion)
  File "/usr/lib/python3.8/functools.py", line 875, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/usr/local/lib/python3.8/dist-packages/parakeet/metadata.py", line 239, in export
    return _export_Config(config, sample, directory, relion)
  File "/usr/local/lib/python3.8/dist-packages/parakeet/metadata.py", line 262, in _export_Config
    export_relion(config, sample, directory)
  File "/usr/local/lib/python3.8/dist-packages/parakeet/metadata.py", line 214, in export_relion
    exporter.write_input_file()
  File "/usr/local/lib/python3.8/dist-packages/parakeet/metadata.py", line 88, in write_input_file
    starfile.write(data, self.input_filename, overwrite=True)
  File "/usr/local/lib/python3.8/dist-packages/starfile/functions.py", line 44, in write
    StarWriter(
  File "/usr/local/lib/python3.8/dist-packages/starfile/writer.py", line 43, in __init__
    self.write()
  File "/usr/local/lib/python3.8/dist-packages/starfile/writer.py", line 67, in write
    self.write_data_blocks()
  File "/usr/local/lib/python3.8/dist-packages/starfile/writer.py", line 80, in write_data_blocks
    write_loop_block(
  File "/usr/local/lib/python3.8/dist-packages/starfile/writer.py", line 181, in write_loop_block
    df = df.map(lambda x: f'{quote_character}{x}{quote_character}' 
  File "/usr/local/lib/python3.8/dist-packages/pandas/core/generic.py", line 5989, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'map'

EDIT: I did get it working for v0.4.3

[jmp1985]

Hi. Apparently for python3.8 an old version of pandas is installed. I am trying to update the docker image to fix the issue but it wasn't as straightforward as I hoped. I will let you know once I've managed to fix the issue!

[jmp1985]

Hi. Just to let you know that I have added a new tag v0.5.3 which I think fixes this issue. Let me know if you have any other issues!

[david-yw-t]

Hi, Thanks for your help! Kind regards, David Sent from Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: James Parkhurst ***@***.***> Sent: Wednesday, August 28, 2024 2:07:27 PM To: rosalindfranklininstitute/parakeet ***@***.***> Cc: david-yw-t ***@***.***>; Mention ***@***.***> Subject: Re: [rosalindfranklininstitute/parakeet] Generating Numbers of Images with Stored Orientations (Discussion #73) Hi. Just to let you know that I have added a new tag v0.5.3 which I think fixes this issue. Let me know if you have any other issues! — Reply to this email directly, view it on GitHub<#73 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANPPIK2U3C4Q6SY4BANP3OLZTW4P7AVCNFSM6AAAAABM7YHBI6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANBXGQ2TANY>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[david-yw-t]

I've been trying to get the generated images at given rotations to match up with my own python code. Strangely, if I set the scan mode to tilt_series, and try rotations around axes (1,0,0),(0,1,0),(0,0,1) I can get everything to match up. But when I switch to single_particle, the rotations don't match up.

Is there a way to export rotation vectors directly, so I can debug?

[jmp1985]

Hi.

Looks like there was a bug. The Euler angles were being generated randomly without reference to the image file. This is why it worked for the tilt series but not single particle scan.

I've changed the parakeet.metadata.export command as follows:

parakeet.metadata.export -c config.yaml -s sample.h5 -i image.h5

This will take the scan from the image metadata directly. Currently I only export a subset of the metadata to the .mrc files due to limitations with that file format so to do this properly requires the .h5 for the moment. I will look at improving the metadata export for mrc though!

I've added a new tag v0.5.4 so try using that version.

You can get the rotation vectors directory from the h5 files as follows:

import h5py
import numpy as np

if __name__ == '__main__':

    handle = h5py.File("exit_wave.h5") # or optics.h5 or image.h5
    axis_x = handle["entry"]["data"]["tilt_axis_x"]
    axis_y = handle["entry"]["data"]["tilt_axis_y"]
    axis_z = handle["entry"]["data"]["tilt_axis_z"]

    rotvec = np.stack([axis_x, axis_y, axis_z]).T

    print(axis)

Thanks for highlighting the issues you've found! Let me know if you find anything else!