Hint Based and Reference Based Image Colourization Algorithm

By -
Hint Based and Reference Based Image Colourization Algorithm

Introduction

The term 'colourization' was coined by Wilson Markle in 1970 to describe a computer-based method he developed for adding colours to black and white TV programs and movies. Colourization is the process of applying colours to a monochrome image or movie. About a century ago, the only source of coloured images was hand paintings. There is always some detail missing in black and white images. Hence to enhance the quality and to generate more attractive and detailed images, the idea of colourization started developing. Earlier the task of colourization was very time-consuming and an expensive job. By the end of the second world war, coloured images were getting a great grip. The typical task required the segmentation of images and then tracking those regions to colourize. However, with advancements in computer science, a few decades back, automated colourization algorithms came into existence. Here we are going to discuss such automated algorithms.

Why there is a need for colourization?      

As mentioned earlier, monochromic images are not as detailed as they should be. Even today, photography beyond the visible spectrum is always monochromic. X-ray images, infrared images, and images from space telescopes, even today are monochromic. The beautiful and colourful images of our mighty universe we see today are not originally coloured. It needs a special team of experts to colourize those thousands and thousands of image data from space. Hence, automated colourization algorithms have always been a centre of attraction for many computer scientists and astronomers.

Hint Based Colourization  

A group of researchers consisting of Anat Levin, Dain Lischinski and Yair Weiss developed an automated colourization technique which does not require segmentation and a complex process. By providing just scribbles on the monochrome image, with the flood-fill technique the colours will automatically fill the image. Also, the problem of colourization at different boundaries of the image is also solved. 

This technique is based on the simple concept that similar colour pixels have similar intensities in the monochrome image, in other words, the nearby pixels in space-time that have similar grey levels should have similar colours. The technique is based on a framework applicable to both, image sequences and still images. Here the user will decide the colour to be applied in form of scribble, and then the algorithm will automatically propagate colours to the remaining pixels. An example is shown below.

Hint Based and Reference Based Image Colourization Algorithm


Here, in the first image the user has applied scribble over the monochrome image. The algorithms will automatically recognize the boundaries and will propagate the colour to the pixels of similar intensities. For example, the boundary between yellow and green colour in the t-shirt of this child is decided precisely by the algorithm. If we try to manually colourize this image, then one needs to first divide the image using segmentation and the colour boundaries must be decided. 

Apart from colouring the monochrome, this algorithm works accurately for recolouring any image. If one needs to change the colour of any object in the image then a scribble of new colour is applied over the object and the algorithm will automatically propagate the new colour pixels. This is shown in the figure below. 

Hint Based and Reference Based Image Colourization Algorithm

  
Here the user wants to recolour the orange with green colour. Hence the green coloured scribbles are applied to the orange. One amazing fact to know here is that the algorithm only recoloured the selected orange, i.e. the object on which scribbles were applied. The orange in the background is untouched. This proves the effectiveness of the algorithm in recognizing the colour boundaries.

Reference Based Colourization 

The hint-based technique is however not the first automated method in the market. Previously various algorithms were developed for the same purpose. One technique by Welsh and Mueller in 2002 colourize the image using a reference image. The algorithm basically transforms colour from the reference image to the target image. This technique also examines the intensity of neighbouring pixels in the target image and transfers the colours from the reference image with similar intensities. Instead of propagating the colour, this algorithm simply transfers the colour.  

This technique works well for images with differently coloured regions, where intensities of pixels have a large difference. But for pixels with minor differences in intensities, the user must manually provide samples indicating similar colours in two images. Also, the selection of reference images must be very precise. The selected reference image must have pixels of required (matching) intensities otherwise the colourization will not generate the required results. An example of this technique is shown below.

Hint Based and Reference Based Image Colourization Algorithm


Here the first image is the reference image and the second one is the target image. Both images are probably of the same mountain range. The only thing to keep in mind is that the reference image must have similar intensity pixels. The red, green and blue boxes are actually samples (swatches) provided for better colourization results. The reference image was captured by Paul Kienitz and the colourized image was captured by Ansel Adam. 

Conclusion

Here we discussed two amazing colourizing methods. Both methods are effective in their own way. Even today, the field of colourization is continuously upgrading. Many new techniques and upgradation of the above two techniques have been developed. Today, colourization is a much easier job than it used to be a few decades back. We have a lot of algorithms that automatically colourize the image. The overhead of segmentation and recombining is reduced today. 


Comments

Post a Comment

Please do not use foul language while commenting. We will appreciate your suggestions and reviews in the comments.

Read Also

Assisting Neuroimaging through DL

By -
Image
Introduction: In recent years, neuroimaging has been a research attraction, especially due to fMRI (Functional Magnetic Resonance Imaging) advancements. The fMRI data comprises of often several individuals. This data is used by researchers to study the association between the cognitive states of an individual and the underlying brain activities. The fMRI data used for neuroimaging is ideally suited for deep learning applications. The large, structured datasets of fMRI can be used for representation-learning methods of the DL. Generally, DL can be defined as a learning method with multiple levels of abstraction, where at each level the input data is transformed by a simple non-linear function which enables the model to recognize complex patterns. With higher-level representation, DL methods can associate a target variable with variable patterns in input data. Also, DL techniques can independently acquire these transformations from the data, eliminating the need for a comprehensive preex
Read more

Marine eDNA Analysis using DL techniques

By -
Image
Introduction Environmental DNA (eDNA) is the genetic information or material released by an organism into its surroundings through activities like skin shedding, mucus, urine or other secretions. These materials contain DNA fragments that can be found in soil, water or air samples. The concept of eDNA is based on the fact that organisms constantly leave organic traces behind them. By analysing the eDNA, researchers can detect the presence of any species in that area. This method is a valuable technique for biodiversity monitoring and ecological research.    Unlike traditional methods that involve direct observation or capturing of organisms, eDNA analysis allows researchers to study biodiversity without directly interacting with the species. This non-invasive approach is particularly useful for studying elusive or endangered species. eDNA analysis has been applied in various ecosystems, including freshwater, marine, and terrestrial environments. It is used to study aquatic organisms, t
Read more

Deep Neural Networks for ADMET properties' prediction

By -
Image
Introduction The absorption, distribution, metabolism, excretion and toxicity properties of any compound contributes to its potential as an orally administered drug. A set of criteria defined by Rule of Five or the Pfizer's Rule of Five stats that any compound meeting these criteria are more likely to have ADMET properties. The importance of these properties has increased in modern medicinal chemistry and new machine learning techniques are employed to predict these properties. The predictive ADMET models have helped in discovery of small molecules with improved safety and dose. Deep Neural Networks are showing usefulness in such predictions due to improved computational efficiency, larger datasets and adaption of image processing in the chemistry. Early Models The early DNNs employed showed improved prediction performance. Many researchers raised concerned that this improvement is in fact has no relation with better computational capacity, rather it is result of mere memorizing of
Read more

How Visual Cortex inspired the Convolutional Neural Networks

By -
Image
Introduction Since the 1960s, after the successful development (at least with respect to the existing technologies of that time) of the first perceptron by Frank Rosenblatt, researchers around the world have been trying to make computers more brain-like. In other words, they are constantly trying to develop programs and hardware that work much like our brains. However, creating a machine that thinks and programs like a human brain is still impossible, given the limitations of present technologies. But, we are successful in this direction up to an agreeable extent. The perceptron developed by Rosenblatt can be considered one of the first implementations of neural networks. It was a huge hardware model, rather than a Python program like today's perceptrons.  Rosenblatt's perceptron was a simple binary input-output system, with various limitations. It was not trainable for different patterns, rather but for simple image recognization. The major drawback was its inability to perfor
Read more