Palette-based Photo Recoloring: Acceleration

Design Discourse Two


The section titled acceleration, describes an acceleration scheme that allows an RBF interpolation scheme to be used in an interactive application.

It states that through this process, the weights are found only around the color and the initial palette colors, therefore, in principle the RBF computations need only be performed when the initial palette is established (not during color palette editing).

This process is then accelerated by storing (caching) this ‘weight’ data so that it is served faster when required in the future.

So now during the color editing stage, when the initial palette colors are known, we can use the ‘weight’ data locations that were previously stored and calcualte the output color.

Finally, to recolor each pixel in the image, trilinear interpolation is used on eight nearest grid values, which is considered to not only give our implementation interactive performance, but also improves any small discontinuities in the output color due to the non-convex gamut.

Palette-based Photo Recoloring: Color Transfer

Design Discourse Two


Whilst reading through the various sections of the article I came across a series of terms and values that initially made understanding the article quite difficult, below are these terms and values defined through external research, so that I may have a better understanding of what the article is explaining.

F- transfer function

L-Ordering of luminance 

FLluminance Transfer function

CSingle Colour

RBFRadial basis function

Transfer functionA mathematical function relating the output or response of a system such as a filter circuit to the input or stimulus.

Gamut: The range of colors that a color device can display or print. A color that may be displayed on your monitor in RGB may not be printable in the gamut of your CMYK printer.

Interpolation- a method of constructing new data points within the range of a discrete set of known data points.

The section indicated as color Transfer, introduces a more complex method of color transfer as seen in the previous section.

The function is know as ‘Faband plays an analogous role in the ‘ab’ channels.

The function (F1) is devised for a situation where the original palette contains a single color. and the user modifies it to be color C1. (As shown below)

color_transfer

Using this method it is desirable to translate colors in the same direction, and its suggests that a naive strategy might simply add exactly the same offset vector.

In contrast, it would be easy to go out of gamut, and simply clamping to the nearest in-gamut value would violate the one-to-one and dynamic range goals. (as seen below)

figure

A naive transfer function that copies the color offset from the palette to every color in the image, even if clamped to remain in gamut, reduces dynamic range in the image (middle). Our transfer function (right) is able to make better use of the dynamic range. Photo courtesy of the MIT-Adobe FiveK Dataset [2011].

Alternatively a scheme is devised that translates colors that are far away from the boundary of the gamut, but squeezes values nearer to the boundary towards the single color.

Below is the equational representation of the scheme mentioned above:

dat_equation

The article continue’s to describe a different scheme, a RBF interpolation scheme.

Equational representation below:

equation-2

This approach leads to smooth interpolation of the individual transfer functions at color 1,unfortunately solving the system of equations set up by the RBFs can lead to negative weights, with two potential hazards. 

  1. It will add some component of the opposite behavior of some palette changes.
  2. More dangerously, it can throw the result out of gamut.

Simple fix to the problem, clamp any negative weights to zero and renormalize the non-zero weights.

However, this RBF interpolation scheme’s naive application to the image would require making this computation for every unique color in the image, despite being fast.

Palette-based Photo Recoloring: Video Recoloring

Design Discourse Two


Video Recoloring

This method is considered to be a fast way to recolour a video, in real time. And resembles the image based coloring method, Palette based Recoloring.

For this application, A source palette needs to be chosen for each frame. The implementation selects a palette from one of the frames and uses it throughout the sequence.

This approach encourages temporal coherence, but might not be suitable for long sequences.

The selection method could be extended to a sliding window scheme, thus making is usable for longer, more heterogeneous sequences.

Palette-based Photo Recoloring: Duotone

Design Discourse Two


What is Duotone reproduction?

  • Duotone reproduction is a traditional printing technique that typically involves two colors of ink applied via halftone patterns over white paper.
  • This style gives an overall sense of coloration and has a nostalgic quality (but costs less than full-color printing involving three or four inks).
  • Digital imaging software like Photoshop provides a duotone function to produce this effect, while allowing the user to choose the ink color(s). Our method can easily produce a similar effect as follows.
  • We start with a grayscale image (or desaturate a color image). We select a 3-color palette and force one of the colors to be white (paper).
  • Finally the user can adjust the two darker (ink) palette colors to achieve various imagery in the style of a duotone.

duotone_1

Palette-based Photo Recoloring: Related Work

Design Discourse Two


Related Work: Palette based Recoloring

Lin:2013 has suggested a method for coloring vector art by palettes based on a probabilistic model.

probabilistic model: Statistical analysis tool that estimates, on the basis of past data, the probability of an event occurring again.

colorpallet

They learn and predict the distribution of properties such as saturation, lightness and contrast for individual regions and adjacent regions, and use the predicted distributions and color compatibility model,O’Donovan:2011, to score pattern colorings.

Palette-based Photo Recoloring: Related Work

Design Discourse Two


Related Work: Edit Propagation (Stroke Based Recoloring)

Stroke-based methods suggest recoloring images by drawing scribbles in a desired color on different regions, automatically propagating these edits to similar pixels.

Levin:2004Pellacini:2008Li and Chen:2009

thewindrisesblog

This is a example of stroke based recoloring that I carried out myself. As shown above, the original image was black and white, however using stroke based recoloring, I applied the desired color to the desired region within the image.

Xuelin Chen:2014, propose sparsity-based edit propagation by computing only on a set of sparse, representative samples instead of the whole image or video. This accelerates and saves memory, especially for high-resolution inputs.

zou_cvpr14

Palette-based Photo Recoloring: Related Work

Design Discourse Two


Related Work: Automatic Color Enhancement

Vladimir Bychkovsky:2011: MIT CSAIL, built a large retouching dataset collected from professional photographers to learn an automatic model for tone adjustment in the luminance channel.

Luminance Channel: The channel through which brightness is measured.

Daniel Cohen-Or:2006:Tel Aviv University, proposed to automatically enhance image colors according to harmonization rules. However, the user can only control the hue template type and rotation, which is not flexible enough for our needs.

harmonization rules: Refers to the method used to enhance the harmony of colors in order to make an image more aesthetically please to a human’s visual perception. 

Hou and Zhang:2007, provide several concepts for users to change the mood of an image. These concepts are extracted by clustering hue histograms for different topics. They only provide 8 concepts, which limits their transformation and editing styles.

Clustering hue histograms: Grouping the numerical data of colour or shade  and displaying it using graphical representation.

Image001

Image: Obtained from wikimedia-hue histograms