Integrate black forest labs FLUX.1 models into Galaxy

tools/flux/.shed.yml

@@ -0,0 +1,14 @@
+name: black_forest_labs_flux
+owner: bgruening
+description: Using the black forest labs FLUX.1 models to generate image based on user input.
+long_description: |
+  This tool uses the black forest labs FLUX.1 models to generate image based on user input.
+  Users can specify the model (dev,schnell),and provide prompt.
+  The tool will then generate and return the corresponding image based on the input provided.
+remote_repository_url: https://github.com/bgruening/galaxytools/tree/master/tools/flux
+homepage_url: https://github.com/bgruening/galaxytools/tree/master/tools/cflux
+type:
+categories:
+  - Machine Learning
+maintainers:
+  - arash77

tools/flux/flux.py

@@ -0,0 +1,32 @@
+import sys
+
+import torch
+from diffusers import FluxPipeline
+
+model = sys.argv[1]
+
+prompt_type = sys.argv[2]
+if prompt_type == "file":
+    with open(sys.argv[3], "r") as f:
+        prompt = f.read().strip()
+elif prompt_type == "text":
+    prompt = sys.argv[3]
+
+if model not in ["black-forest-labs/FLUX.1-dev", "black-forest-labs/FLUX.1-schnell"]:
+    print("Invalid model!")
+    sys.exit(1)
+
+
+pipe = FluxPipeline.from_pretrained(model, torch_dtype=torch.bfloat16)
+pipe.enable_sequential_cpu_offload()
+pipe.vae.enable_slicing()
+pipe.vae.enable_tiling()
+pipe.to(torch.float16)
+
+image = pipe(
+    prompt,
+    num_inference_steps=4,
+    generator=torch.Generator("cpu").manual_seed(42),
+).images[0]
+
+image.save("output.png")

tools/flux/flux.xml

@@ -0,0 +1,100 @@
+<tool id="black_forest_labs_flux" name="FLUX" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" profile="23.0">
+    <description>text-to-image model</description>
+    <macros>
+        <token name="@TOOL_VERSION@">2024</token>
+        <token name="@VERSION_SUFFIX@">0</token>
+    </macros>
+    <requirements>
+        <requirement type="package" version="3.12">python</requirement>
+        <requirement type="package" version="2.4.1">pytorch</requirement>
+        <requirement type="package" version="0.19.1">torchvision</requirement>
+        <requirement type="package" version="0.30.2">diffusers</requirement>
+        <requirement type="package" version="4.44.2">transformers</requirement>
+        <requirement type="package" version="0.34.0">accelerate</requirement>
+        <requirement type="package" version="0.2.0">sentencepiece</requirement>
+        <requirement type="package" version="4.25.3">protobuf</requirement>
+        <requirement type="package" version="0.24.6">huggingface_hub</requirement>
+    </requirements>
+    <command detect_errors="exit_code"><![CDATA[
+export HF_HOME='$flux_models.fields.path' &&
+python '$__tool_directory__/flux.py'
+'$flux_models'
+'$input_type_selector'
+'$prompt'
+    ]]></command>
+    <configfiles>
+    </configfiles>
+    <inputs>
+        <param name="flux_models" label="Model data" type="select" help="contact the administrator of this Galaxy instance if you miss model data">
+            <options from_data_table="huggingface">
+                <filter type="static_value" column="4" value="flux"/>
+                <filter type="static_value" column="5" value="1"/>
+            </options>
+            <validator message="No model annotation is available for FLUX" type="no_options"/>
+        </param>
+        <conditional name="input_type">
+            <param name="input_type_selector" type="select" label="Choose the type of input">
+                <option value="file" selected="true">File based input</option>
+                <option value="text">Text based input</option>
+            </param>
+            <when value="file">
+                <param name="prompt" type="data" optional="false" format="txt" label="Prompt file" help="This data will be used as prompt"/>
+            </when>
+            <when value="text">
+                <param name="prompt" type="text" optional="false" label="Prompt" help="This text will be used as prompt" area="true">
+                    <validator type="empty_field"/>
+                </param>
+            </when>
+        </conditional>
+    </inputs>
+    <outputs>
+        <data name="output" format="png" label="${tool.name} on ${on_string}" from_work_dir="./output.png"/>
+    </outputs>
+    <tests>
+        <test expect_exit_code="1" expect_failure="true">
+            <param name="flux_models" value="unknown"/>
+            <param name="input_type_selector" value="file"/>
+            <param name="prompt" value="flux_test.txt"/>
+            <assert_stdout>
+                <has_text text="Invalid model!"/>
+            </assert_stdout>
+        </test>
+    </tests>
+    <help><![CDATA[
+
+.. class:: infomark
+
+**What it does**
+
+FLUX is a text-to-image model that generates images from textual descriptions.
+
+Usage
+.....
+
+**Input**
+
+1. **Model data**:
+    Select the model data from the dropdown list. The available models are dev and schnell.
+    For using dev model, make sure to not use it for commercial purposes.
+
+2. **Input Prompt**: You can provide the input prompt in two ways:
+    - **File based input**: Upload a txt file containing the prompt.
+    - **Text based input**: Enter the prompt text in the text area.
+
+**Output**
+
+This response is saved in the `output.png` file.
+
+    ]]></help>
+    <citations>
+        <citation type="bibtex">
+            @misc{flux,
+                author = {black forest labs},
+                title = {FLUX github repository},
+                howpublished = {\url{https://github.com/black-forest-labs/flux}},
+                year = {2024},
+                note = {Accessed: 2024-09-04}
+            }
+        </citation>
+    </citations>
+</tool>

tools/flux/test-data/flux_test.txt

@@ -0,0 +1 @@
+Extreme close-up of a spiral galaxy, direct frontal view, with highly detailed stars and nebulae. The image has sharp focus on texture and color, with ethereal cosmic lighting to capture the galaxy's natural shine and depth. The words 'Galaxy Project' are painted in large, white brush strokes with visible texture, centered across the image.

tools/flux/tool-data/huggingface.loc.sample

@@ -0,0 +1,18 @@
+
+#This is a sample file distributed with Galaxy that is used to define huggingface
+#models, using 7 columns tab separated
+#(longer whitespace are TAB characters):
+#
+#The entries are as follows:
+#
+#<unique_build_id>	<display_name>	<pipeline_tag>	<domain>	<free_tag>	<version>	<folder_base_path>
+#
+#pipeline_tag (see this URL: https://huggingface.co/models?pipeline_tag=text-to-image they call it that way)
+#domain (defined by the Galaxy community, e.g. image / sequence / text)
+#free_tag (not idea yet, but could be freely used by an admin to specify more filter options)
+#version (if available)
+#Your huggingface.loc file should include the huggingface cached model folder you have stored.
+#For example:
+#
+#black-forest-labs/FLUX.1-dev	FLUX.1 [dev] is an open-weight, guidance-distilled model for non-commercial applications.	text-to-image	image	flux	1	/path/to/huggingface/that-contains/hub/models--black-forest-labs--FLUX.1-dev
+#black-forest-labs/FLUX.1-schnell	FLUX.1 [schnell] is the fastest model, tailored for personal use.	text-to-image	image	flux	1	/path/to/huggingface/that-contains/hub/models--black-forest-labs--FLUX.1-schnell