Skip to content

mkazhdan/TextureSignalProcessing

BranchesTags

Repository files navigation

Gradient Domain Texture Processing (Version 4.50)

links executables usage compilation changes support
This software supports gradient-domain signal processing within a texture atlas. Supported applications include:
  • (localized) texture smoothing and sharpening,
  • vector-field visualization akin to line-integral convolution,
  • computation of single-source geodesics, and
  • simulation of reaction-diffusion following the Gray-Scott model.
LINKS
EXECUTABLES
    TextureFiltering: Supports the (localized) smoothing and sharpening of a texture by solving a screened Poisson equation which gives the signal whose values match the input and whose gradients match the modulated gradients of the input. If no output texture is specified, the executable will launch an interactive viewer that supports local "painting" of gradient modulation values and prescription of a global interpolation weight.
    In the interactive viewer the modulation can be set globally by dragging the slider on the top left.
    The modulation can be set locally by holding the [SHIFT] key down and either dragging with the left mouse button (to smooth) or the right mouse button (to sharpen).
    --in <input mesh and texture names>
    These two strings specify the the names of the mesh and the texture image.
    The input mesh is assumed to be in PLY format, giving the set of vertices with the x-, y-, and z-coordinates of the positions encoded by the properties x, y, and z the set of polygons encoded by two lists. The first gives the indices of the vertices in the polygon (integers). The second gives the texture coordinates at each polygon corner (pairs of floats).
    The input texture is assumed to be an image if the file extension is png, jpg, or jpeg, and a normal map if the extension is normap.
    [--out <output texture>]
    This string is the name of the file to which the processed texture will be written.
    [--outVCycles <output v-cycles>]
    This integer specifies the number of v-cycles to use if the processed texture is output to a file and a direct solver is not used. The default value for this parameter is 6.
    [--interpolation <interpolation weight>]
    This floating point values gives the interpolation weight.
    The default value for this parameter is 1000.
    [--modulation <gradient modulation>]
    This floating point values gives the (uniform) gradient modulation.
    The default value for this parameter is 1.
    [--jitter <random seed>]
    If specified, this integer value is used to seed the random number generation for jittering. (This is used to avoid singular situations when mesh vertices fall directly on edges in the texture grid. In such a situation, the executable will issue a warning "Zero row at index ...".)
    [--useDirectSolver]
    If enabled, this flag specifies that a direct solver should be used (instead of the default multigrid solver).
    TextureStitching: Supports the stitching together of multiple (possibly overlapping) textures by solving a screened Poisson equation with value constraints defined by the input texture. The interactive viewer runs in two modes:
    1. A user specifies a single composite texture and mask file indicating when texels in the composite come from the same source. In this case gradient constraints are obtained by copying gradients from the composite whenever the two texels defining an edge come from the same source, and setting the gradient constraint to zero along edges coming from different sources. The viewer shows the stitched texture on the left and the composite texture on the right.
    2. A user specifies multiple partial texture files and corresponding confidence masks. In this case gradient constraints are obtained by blending gradients from the different inputs, weighted by confidence, and setting gradients to zero in regions where there are no textures with non-zero confidence. The viewer shows the stitched texture on the left and a partial texture on the right. The user can selectively replace blended value/gradient constraints with the values/gradients from the partial texture by holding the [SHIFT] key down and dragging over the region to be in-painted.
    --in <input mesh, composite texture, and mask>
    These three strings specify the the names of the mesh, the texture image, and the mask image.
    The input mesh is assumed to be in PLY format, giving the set of vertices with the x-, y-, and z-coordinates of the positions encoded by the properties x, y, and z the set of polygons encoded by two lists. The first gives the indices of the vertices in the polygon (integers). The second gives the texture coordinates at each polygon corner (pairs of floats).
    The input texture and mask are assumed to be images in png, jpg, or jpeg format. Black pixels in the mask file should be used to denote regions where the texel value is unkown.
    --in <input mesh, texture format specifier, and confidence format specifier>
    These three strings specify the the names of the mesh, the format string for the texture images, and the format string for the confidence images.
    The input mesh is assumed to be in PLY format, giving the set of vertices with the x-, y-, and z-coordinates of the positions encoded by the properties x, y, and z the set of polygons encoded by two lists. The first gives the indices of the vertices in the polygon (integers). The second gives the texture coordinates at each polygon corner (pairs of floats).
    The input textures and confidence maps are assumed to be images in png, jpg, or jpeg format.
    For the texture and confidence names to be interpreted as format specifiers, the --multi flag must be specified.
    [--out <output texture>]
    This string is the name of the file to which the stitched texture will be written.
    [--outVCycles <output v-cycles>]
    This integer specifies the number of v-cycles to use if the stitched texture is output to a file and a direct solver is not used.
    The default value for this parameter is 6.
    [--interpolation <interpolation weight>]
    This floating point values gives the interpolation weight.
    The default value for this parameter is 100.
    [--dilateBounaries <dilation radius>]
    This integer values gives the radius by which the boundaries of the segments should be dilated before stithing is performed.
    The default value for this parameter is -1, indicating no dilation.
    [--jitter <random seed>]
    If specified, this integer value is used to seed the random number generation for jittering. (This is used to avoid singular situations when mesh vertices fall directly on edges in the texture grid. In such a situation, the executable will issue a warning "Zero row at index ...".)
    [--useDirectSolver]
    If enabled, this flag specifies that a direct solver should be used (instead of the default multigrid solver).
    [--multi]
    If enabled, this flag specifies that the second and third arguments to the --in parameter are to be interpreted as format specifiers for the textures confidence map files.
    LineIntegralConvolution: Supports the line-integral-convolution visualization of a vector-field through anisotropic diffusion by defining a new metric on the surface that stretches distances along the vector-field values, diffusing a random color texture with respect to the new anisotropic metric, and then sharpening the resulting signal. If no output texture is specified, the executable will launch an interactive viewer that supports iteratively stepping through the diffusion.
    Hit [SPACE] to start the iterative solver or hit "+" to advance one iteration at a time.
    --in <input mesh name>
    This string specifies the name of the mesh.
    The input mesh is assumed to be in PLY format, giving the set of vertices with the x-, y-, and z-coordinates of the positions encoded by the properties x, y, and z the set of polygons encoded by two lists. The first gives the indices of the vertices in the polygon (integers). The second gives the texture coordinates at each polygon corner (pairs of floats).
    [--inVF <vector-field file>]
    This string specifies the file containing the vector-field for visualization. (If this parameter is not specified, the principal curvature direction is used.)
    This file is assumed to be in binary, with the first four bytes storing an integer representing the number of vectors (this should be equal to the number of triangles in the mesh) followed by the list of vectors. The latter are encoded using double-precision floating point values and should be 8*num_triangles*dim bytes, with num_triangles the number of triangles/vectors and dim the dimension of vector-field. (The value of dim is equal to two if the --intrinsicVF is specified an three otherwise.)
    [--intrinsicVF]
    If enabled and a vector-field is specified, this flag indicates that the vector values are represented with two values per vector, using an intrinsic frame. Specifically, for triangle ( v0 , v1 , v2 ), the two-dimensional coefficients ( x , y ) correspond to the three-dimensional tangent vector ( x·(v1-v0) , y·(v2-v0) ).
    [--out <output texture>]
    This string is the name of the file to which the line-integral-convolution texture will be written.
    [--outVCycles <output v-cycles>]
    This integer specifies the number of v-cycles to use if the processed texture is output to a file and a direct solver is not used. The default value for this parameter is 10.
    [--licInterpolation <line-integral-convolution interpolation weight>]
    This floating point values gives the interpolation weight used for the line-integral-convolution.
    The default value for this parameter is 10000.
    [--sharpInterpolation <sharpening interpolation weight>]
    This floating point values gives the interpolation weight used for sharpening the line-integral-convolution results.
    The default value for this parameter is 10000.
    [--modulation <sharpening gradient modulation>]
    This floating point values gives the gradient modulation used for sharpening the line-integral-convolution results.
    The default value for this parameter is 100.
    [--width <output texture width>]
    This integers specifies the width of the output texture. The default value for this parameter is 2048.
    [--height <output texture height>]
    This integers specifies the height of the output texture. The default value for this parameter is 2048.
    [--jitter <random seed>]
    If specified, this integer value is used to seed the random number generation for jittering. (This is used to avoid singular situations when mesh vertices fall directly on edges in the texture grid. In such a situation, the executable will issue a warning "Zero row at index ...".)
    [--minor]
    If enabled, this flag specifies that the directions of minimal principal curvature should be used to define the vector-field (instead of the default maximal principal curvature directions).
    [--useDirectSolver]
    If enabled, this flag specifies that a direct solver should be used (instead of the default multigrid solver).
    Geodesics: An interactive tool for visualization of single-source geodesics using the heat method.
    In the interactive viewer the source can be set by holding the [SHIFT] key down and clicking/dragging with either mouse button.
    --in <input mesh name>
    This string specifies the the name of the mesh.
    The input mesh is assumed to be in PLY format, giving the set of vertices with the x-, y-, and z-coordinates of the positions encoded by the properties x, y, and z the set of polygons encoded by two lists. The first gives the indices of the vertices in the polygon (integers). The second gives the texture coordinates at each polygon corner (pairs of floats).
    [--interpolation <diffusion interpolation weight>]
    This floating point values gives the interpolation weight used for diffusing the initial delta function.
    The default value for this parameter is 10000.
    [--width <output texture width>]
    This integers specifies the width of the output texture. The default value for this parameter is 1024.
    [--height <output texture height>]
    This integers specifies the height of the output texture. The default value for this parameter is 1024.
    [--jitter <random seed>]
    If specified, this integer value is used to seed the random number generation for jittering. (This is used to avoid singular situations when mesh vertices fall directly on edges in the texture grid. In such a situation, the executable will issue a warning "Zero row at index ...".)
    [--useDirectSolver]
    If enabled, this flag specifies that a direct solver should be used (instead of the default multigrid solver).
    ReactionDiffusion: Performs simulation of reaction-diffusion based on the Gray-Scott model. If no output texture is specified, the executable will launch an interactive viewer that supports iteratively stepping through the reaction-diffusion process.
    Hit [SPACE] to start the reaction-diffusion process or hit "+" to advance one step at a time.
    --in <input mesh name>
    This string specifies the the name of the mesh.
    The input mesh is assumed to be in PLY format, giving the set of vertices with the x-, y-, and z-coordinates of the positions encoded by the properties x, y, and z the set of polygons encoded by two lists. The first gives the indices of the vertices in the polygon (integers). The second gives the texture coordinates at each polygon corner (pairs of floats).
    [--out <output texture>]
    This string is the name of the file to which the reaction-diffusion texture will be written.
    [--outSteps <output reaction-diffusion steps>]
    This integer specifies the number of reaction-diffusion steps to be taken. The default value for this parameter is 1000.
    [--width <output texture width>]
    This integers specifies the width of the output texture. The default value for this parameter is 512.
    [--height <output texture height>]
    This integers specifies the height of the output texture. The default value for this parameter is 512.
    [--jitter <random seed>]
    If specified, this integer value is used to seed the random number generation for jittering. (This is used to avoid singular situations when mesh vertices fall directly on edges in the texture grid. In such a situation, the executable will issue a warning "Zero row at index ...".)
    [--useDirectSolver]
    If enabled, this flag specifies that a direct solver should be used (instead of the default multigrid solver).
    [--dots]
    If enabled, this flag specifies that the feed/kill parameters for dot-formation should be used. Otherwise, the feed/kill parameters for stripes are used.
USAGE EXAMPLES (WITH SAMPLE DATA)
For testing purposes, a number of textured mapped models are provided (using the .ply extension). Of these, David and Julius include normal maps (using the .normap extension), Fertility includes the eight harmonic vector-fields (using the .vf extension), and Rooster uses (partial) texture maps as well as a mask image and confidence maps.
    TextureFiltering To run this executable you must specify the input mesh as well as the texture itself:
    % Bin/*/TextureFiltering --in ../TSP.Data/David/david.ply ../TSP.Data/David/david.normap
    This opens a viewer allowing the user to prescribe both global gradient modulation weights (through the slider) and local modulation weights (through a paint-brush interface, by depressing the [SHIFT] key and dragging with the left mouse button to smooth and the right mouse button to sharpen).
    You can also bypass the viewer and output a globally sharpened/smoothed texture to a file:
    % Bin/*/TextureFiltering --in ../TSP.Data/Julius/julius.ply ../TSP.Data/Julius/julius.normap --out julius.smooth.normap --modulation 0 --interpolation 100
    Here a modulation weight less than 1 indicates that gradients should be dampened (resulting in smoothing) and a small interpolation weight reduces the interpolation penalty, exaggerating the smoothing.
    TextureStitching This viewer can be run in one of two modes:
    1. In addition to the input mesh, specify a (single) composite texture and mask. If adjacent texels share the same mask color, they are assumed to come from the same source, and the gradient between them is preserved. Otherwise, the gradient is set to zero. Additionally, a mask color of black is reserved to indicate that the texel value is unknown.
      For example, running
      % Bin/*/TextureFiltering --in Rooster/rooster.ply ../TSP.Data/Rooster/texels.png ../TSP.Data/Rooster/mask.png
      opens a viewer showing the stitched texture on the left and the composite texture on the right.
    2. In addition to the input mesh, specify (multiple) partial textures and associated confidence maps. The code blends the gradients in regions of overlap, with weights determined by the mask. Texel and confidence file names are specified using integer format specifiers, with zero-indexing. Colors are transformed to scalar confidence values by computing the gray-scale value and normalizing to the range [0,1].
      For example, running
      % Bin/*/TextureFiltering --in Rooster/rooster.ply ../TSP.Data/Rooster/texels-%02d.png ../TSP.Data/Rooster/mask-%02d.png --multi
      opens a viewer showing the stitched texture on the left and the first partial textures on the right.
      Pressing the 't' key toggles forward through the partial textures and pressing 'T' toggles backwards.
      Holding [SHIFT] and clicking on the stitched model replaces the blended gradients under the paint-brush with the gradients from the currently visualized partial-texture.
    You can also bypass the viewer and output the stitched texture to a file:
    % Bin/*/TextureStitching --in Rooster/rooster.ply ../TSP.Data/Rooster/texels-%02d.png ../TSP.Data/Rooster/mask-%02d.png --multi --out stitched.png
    LineIntegralConvolution To run this executable you must specify the input mesh:
    % Bin/*/LineIntegralConvolution --in ../TSP.Data/Fertility/fertility.ply
    This opens a viewer visualizing a vector-field by performing anisotropic diffusion to simulate line-integral-convolution. (To start the iterative solver, press the [SPACE] key.) By default, the vector-field used is defined by the (maximal) principal curvature directions.
    You can also explicitly prescribe the vector-field:
    % Bin/*/LineIntegralConvolution --in ../TSP.Data/Fertility/fertility.ply --inVF ../TSP.Data/Fertility/harmonic-001.vf --intrinsicVF
    (The --intrinsicVF flag is required because the vector-field in the file is represented using two intrinsic coordinates per triangle instead of three extrinsic ones.)
    You can also bypass the viewer and output the line-integral-convolution texture to a file:
    % Bin/*/LineIntegralConvolution --in ../TSP.Data/Hand/hand.ply --minimal --out hand.minimal.jpg
    Here a visualization of the minimal principal curvature directions is written out as a texture image.
    Geodesics To run this executable you must specify the input mesh:
    % Bin/*/Geodesics --in ../TSP.Data/Bunny/bunny.ply
    This opens a viewer allowing the user to prescribe the source of the geodesic by holding the [SHIFT] button and clicking on the source location with either mouse button.
    ReactionDiffusion To run this executable you must specify the input mesh:
    % Bin/*/ReactionDiffusion --in ../TSP.Data/Camel/camel.ply
    This opens a viewer visualizing the "stripes" reaction-diffusion process. (To start the process, press the [SPACE] key.)
    You can also bypass the viewer and output the reaction-diffusion texture to a file:
    % Bin/*/ReactionDiffusion --in ../TSP.Data/David/david.ply --out david.dots.jpg --dots --outSteps 2000
    Here a "dots" pattern is written out to an image. (Empirically, we have found that this reaction-diffusion process takes more steps to converge, hence the larger number of steps.)
COMPILATION AND EXECUTION
  • The Windows executables require both the glew and glut dynamically linked libraries to run. These can be found here and should be included either in the directory with the executables, or in the directory from which the executables are run.
  • Compiling under Windows requires both the glew and glut libraries. These can be found here and should be placed in the output directory for linkage.
HISTORY OF CHANGES

Version 2.00:

  • Added support for reaction-diffusion based on the Gray-Scott model.

Version 3.00:

  • Added support for texture stitching.

Version 4.00:

  • Added Makefile.no_visual to allow building texture filtering/stitching applications without visualizations.

Version 4.01:

  • Added support for reading .obj files.

Version 4.02:

  • Added support for mask visualization.
  • Switched exceptions to warnings.

Version 4.03:

  • Added support for segment boundary dilation in the TextureStitching code.

Version 4.05:

  • Modified the --jitter flag to take a random seed.

Version 4.06:

  • Added support for visualizing weights when using multi-stitching.

Version 4.07:

  • Added support for providing a separate low-frequency signal for texture processing.

Version 4.08:

  • Removing numerical issues in loop construction.

Version 4.50:

  • Code clean-up
SUPPORT
This work genersouly supported by NSF grant #1422325.

About

Gradient-Domain Processing within a Texture Atlas

Resources

Readme
Activity

Stars

71 stars

Watchers

7 watching

Forks

12 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages