Texture Mixing by Interpolating
Deep Statistics via Gaussian Models

Zi-Ming Wang, Gui-Song Xia, Yi-Peng Zhang

[arXiv-preprint] [code coming soon]

Abstract

Recently, enthusiastic studies have devoted to texture synthesis using deep neural networks, because these networks excel at handling complex patterns in images. In these models, second-order statistics, such as Gram matrix, are used to describe textures. Despite the fact that these model have achieved promising results, the structure of their parametric space is still unclear, consequently, it is difficult to use them to mix textures. This paper addresses the texture mixing problem by using a Gaussian scheme to interpolate between second-order statistics. More precisely, we first reveal that the statistics used in existing deep models can be unified using a stationary Gaussian scheme. We then present a novel algorithm to mix these statistics by interpolating between Gaussian models using optimal transport. We further apply our scheme to Neural Style Transfer, where we can create mixed styles. The experiments demonstrate that our method can achieve state-of-the-art results.

Experiments

A. Texture Mixing

We mix pairs of exemplar textures with different weights ρ, when ρ=0 the results is synthesizing the 1st exemplar, and when ρ=1 the results is synthesizing the 2nd exemplar, the results with weight ρ in between are mixtures of these two textures. For each pair of textures, we compare our results (first row) with ImageMelding [1] (second row).

Exemplar Textures


ρ=0 ρ=1/9 ρ=2/9 ρ=3/9 ρ=4/9 ρ=5/9 ρ=6/9 ρ=7/9 ρ=8/9 ρ=9/9

B. Style morphing

We mix pairs of style images different weights ρ, and we generate stylish photos with the mixed styles. For each pair of styles, we compare our results (first row) with Dumoulin's methods[2] (second row).

Original photo Styles


Style 1 ρ=0 ρ=1/9 ρ=2/9 ρ=3/9 ρ=4/9 ρ=5/9 ρ=6/9 ρ=7/9 ρ=8/9 ρ=9/9 Style 2

References

  • S. Darabi, et.al., “Image melding: Combining inconsistent images using patch-based synthesis.” TOG, vol. 31, no. 4, pp. 82–101, 2012
  • V. Dumoulin, et.al., “A learned representation for artistic style,” CoRR, abs/1610.07629, vol. 2, no. 4, p. 5, 2016


Citation

@article{TextureMixDeepG_WXZ_2018,
	  title={Texture Mixing by Interpolating Deep Statistics via Gaussian Models},
	  author={Wang, Zi-Ming and Xia, Gui-Song and Zhang, Yipeng https://arxiv.org/pdf/1807.11035},
	  year={2018},
    }
	

Contact

If you have any problem, please contact: