Image Classification - Black and White, Color

One of the classic problems is classifying the images. There will be series of posts on this topic which will be closely related to machine learning concepts. Though there are different techniques available in the scientific community and market, let’s start with the basic method of classification - decision tree. Look at the dataset,

Channel	Variance	Image Type
Monochrome	low	BW
RGB	low	BW
RGB	High	Color

There is only two types of classification ‘Black and White’ or ‘Color’

MATLAB CODE:

%Image Classification: BW and Color Image

%List of 'jpeg' images

fname = dir('*.jpeg');

for k=1:size(fname,1)

   

    %Read the image

    Img = imread(fname(k).name);

    %Size of the Image

    mn=size(Img,1)*size(Img,2);

   

    %Estimate the difference along the Channels

    VarI=std(double(Img),0,3);

    BelowT = sum(VarI(:)<=25);

    Prob_BT= BelowT/mn;

   

    if(Prob_BT==1)

        figure(1),imshow(Img);title('Black and White Image');

    else

         figure(2),imshow(Img);title('Color Image');

    end

   

   

  

end

The key feature is the ‘Variance’ that ensures how different each value along the Red, Green and Blue Channels at each pixel location.

For instance, at a particular pixel location if Red channel = 30 , Green channel = 30 and Blue channel = 30 then the variance will be zero which infers that the image is grayscale or BW. Also, we can tend to infer that the image as black and white even when the pixel values in these channels are close to each other.

If we look at the car image and its corresponding variance map, we can see that the pixel values of the RGB channels at a particular pixel location varies greatly and thus the variance map shows this huge difference.  We can say that the picture has been classified correctly.

All the RGB channels have the same value with respect to each pixel location which indicates that the image is gray scale. And variance map indicates zero for the whole image. So the Cat image is BW type.

This zebra image which is a natural black and white image even though its RGB components are not same but they are close to each other and the variance map has done a good job to check the closeness between the RGB channels and it has been classified as BW.

The last one is the swan image. On a first glance it looks like a black and white image and I would like to classify it as BW but the variance map tells a different story. We can see the beak region has a high variance where the RGB components differ greatly while the remaining part of the image is almost having RGB components close to each other. 

The variance map has been quite useful feature but maybe we need to classify the image not as ‘Color Image’ but ‘Black and White dominant Image’

EXAMPLE 2:

Add more features and more classification to make the data set little bit complicated and messy. We have obtained the decision tree for the below mentioned data set from the article on ‘DecisionTree – supervised Learner’. Let’s use it to check out the results.

Channel	Variance	DominantB	ImageType
Monochrome	low	High	Black
Monochrome	low	low	White
Monochrome	low	Medium	BW
RGB	low	High	Black
RGB	low	low	White
RGB	low	Medium	BW
RGB	High	High	Color
RGB	High	low	Color

MATLAB CODE:

%Image Classification: BW, Black Dominant Image, White Dominant Image, Color Image

%List of 'jpeg' images

fname = dir('*.jpeg');

for k = 1:size(fname,1)

   

    %Read the image

    Img = imread(fname(k).name);

    %size of the image

    mn = size(Img,1)*size(Img,2);

   

    %Compute Variance

     VarI = std(double(Img),0,3);

     BelowT = sum(VarI(:)<=14);

     Prob_BT = BelowT/mn; %Number of values below the Threshold

       

       

      T1 = (Img(:,:,1) < 100&Img(:,:,2) < 100&Img(:,:,3) < 100);%Black or gray pixels

      T2 = (Img(:,:,1) > 150&Img(:,:,2) > 150&Img(:,:,3) > 150);%White pixels

      

      T1 = (sum(T1(:)) / mn)*100;

      T2 = (sum(T2(:)) / mn)*100;

     

      if((T1/T2) > 0.8 & (T1/T2) < 1.6) %Medium Criteria for the feature 'DominantB'

        

              figure(1),imshow(Img);title('Image Type:Black and White Image','FontSize',20);

             

         

         

      %When the number of black pixels are high   

      elseif(T1 > T2) %High Criteria for the feature 'DominantB'

          %Variance Low

          if(Prob_BT > 0.6)

              figure(1),imshow(Img);title('Image Type:Black Dominant Image','FontSize',20);

            

          else

              figure(1),imshow(Img);title('Image Type:Color Image','FontSize',20);

             

          end

       %When the number of white pixels are high  

      elseif(T2 > T1)%Low Criteria for the feature 'DominantB'

          %Variance Low

          if(Prob_BT > 0.6)

              figure(1),imshow(Img);title('Image Type:White Dominant Image','FontSize',20);

              

          else

              figure(1),imshow(Img);title('Image Type:Color Image','FontSize',20);

              

          end

      end

    %To visualize each image

    pause

   

end

EXPLANATION:

The number of gray or dark pixels is estimated. Similarly, the number of white or light pixels is estimated. If the ratio between these two is almost equal then the image is black and white. If the gray pixels are more than the white pixels then estimate the variance. Based on the variance map, decide whether the image is color image or black dominant image. In the similar manner, if the white pixels are more than the gray pixels then based on the variance map decide whether the image type is color or white dominant image. The result below illustrates the classification done on the images and how they were classified.

The images used for the classification were taken from website the https://pixabay.com/

How to Save/Export Images in MATLAB

             

           This blog illustrates some simple stuff to deal with, when the images need to be saved with certain constraints.

Let’s start with simple text figure that needs to be saved in image format.

figure,axis off;

text(0.35,0.5,'\color[rgb]{red} E \color[rgb]{green}=\color[rgb]{blue} mc^2','FontSize',60,'FontName','Times New Roman','FontAngle','italic');

To save the above figure with different background:

The default background color is set to white in MATLAB. In order to have the different background color as in the figure, select ‘Export Setup’ in the figure window and select ‘Rendering’ from properties then unselect the custom color and finally select ‘Apply to figure’. Then select ‘Save As’ to save the image.

The step by step screenshots are as given below:

The final image saved in the local directory is as shown below.

To save the figure with black color background:

Select the custom color and type ‘k’ in the text box and select ‘Apply to figure’.

For Red, type ‘r’ or [1,0,0]. For Blue , type ‘b’ or [0,0,1]. For Green, type ‘g’ or [0 ,1,0]; For cyan ‘c’ or [0.5,1,1] and so on.

NOTE: ‘r’ denotes red color or a 1 x 3 Vector representing [R,G,B] can be used.  The value ranges between 0 and 1. [1,0,0] denotes value of red component is 1 whereas Green and Blue components are zero. In other words, the color Red is chosen.

The final image saved in the local directory is as shown below.

Example 2:

A=imread('cameraman.tif');

figure,imagesc(A);colormap(jet);

To save the image without the padded white space:

Select ‘Export Setup’ from File in the figure window and select ‘Size’ from properties, type the width and height of the image and change the units to ‘points’. Select  ‘Expand the axes to fill figure’ checkbox and finally select ‘Apply to figure’ to view the changes.

Now save the image.

Image saved in the local directory.

To save the image without the axes:

MATLAB code:

A=imread('cameraman.tif');

figure,imagesc(A);colormap(gray);

set(gca, 'XTickLabelMode', 'manual', 'XTickLabel', [],'YTickLabelMode', 'manual', 'YTickLabel', [],'Xtick',[],'Ytick',[]);

After executing the above code, follow the same procedure as the previous example to save the image without white spaces.

The image can be saved by selecting ‘Export’ tab as well.

Image stored in the local directory.

Read, process and save video in MATLAB

               This tutorial will come in handy if you are interested in learning about video processing using MATLAB. Techniques such as Reading and writing a video file, displaying the frames and writing the frames as images in a folder are discussed below.

To read a video file:

Vptr = VideoReader('VIDE01.mp4')

‘VideoReader’ command in MATLAB creates reader object for the video file. This object contains the metadata or the parameters related to the video such as Frame rate, Height and Width of the frames,duration of the video etc.

To read all the frames in the video, we can use two methods. The first method is to find the number of frames in the video and read it. The second method is to read the frames until no more video frames are available. Here in the tutorial, the frame rate is assumed to be constant throughout the duration of the video. At constant frame rate, the number of frames in the video is obtained by direct multiplication of frame rate and video duration. In our example, the video is 13.29 seconds long and the frame rate is 24.0. Multiplying 13.29 and 24 gives 318.96 that is 319 frames available in the video.

1. To Read the frames in the video , display , save as image file and store as mat file

EXAMPLE : 1

%MATLAB CODE:

Vptr = VideoReader('VIDE01.mp4');

%Num_Frames = Vptr.NumberOfFrames;

NFrames = round(Vptr.FrameRate*Vptr.Duration);

%Find the height and weight of the frame

Nrows = Vptr.height;

Ncols = Vptr.width;

%Preallocate the matrix

Img_s = zeros([Nrows,Ncols,NFrames]);

for i = 1:NFrames

%Read each frame

Img = readFrame(Vptr);

%To display all the frames

figure,imshow(Img);

%To save the images

Img_name=['Image',num2str(i),'.jpg'];

imwrite(Img,Img_name);

%To store in MAT file

Img_s(:,:,i)=Img;

end

%Save the matrix as .mat file

Save Video_Images.mat Img_s;

EXPLANATION:

The above MATLAB code can

a. Display all the frames in the video

b. Save all the frames as images in the current working directory

c. Store all the frames as images in a multidimensional matrix and save it as ‘.mat’ file

After creating the video reader object, the number of frames is computed using the frame rate and duration of the video. The height and width of the frame can be obtained from the metadata.

‘readFrame’ extracts each frame sequentially in the image format. The image can be further displayed using ‘imshow’ or written to an image file using ‘imwrite’ command or stored in a multidimensional matrix as stack of images.

The name format of the images saved in the current working directory will be ‘Image1.jpg’,’Image2.jpg’…’Image319.jpg’

2. To read all the frames in the video and display few frames

EXAMPLE 2:

%MATLAB CODE

Vptr = VideoReader('VIDE01.mp4');

NFrames = round(Vptr.FrameRate*Vptr.Duration);

Jump_ptr = 27;

N = 1;

%To display the Images

for i=1:NFrames

Img = readFrame(Vptr);

if(mod(i-1,Jump_ptr)==0) 

figure(2),subplot(3,4,N),imshow(Img);

              N=N+1;

end

end

EXPLANATION:

The above MATLAB code reads all the frames in the video but displays only few frames. This example typically highlights the use of MATLAB command ‘subplot’.

Instead of displaying all the frames, frames with specific interval are displayed. In this instance,frame 1 will be displayed first, frame 28 the next then followed by 55 and so on and so forth. ‘mod’ command is used to find the remainder after division, so whenever ‘i’ assumes the value equal to multiples of the variable ‘Jump_ptr’ then the image will be displayed. To displayall the images in the same figure, ‘subplot’ can be used.

‘subplot(3,4,N)’ refers that the ‘figure(2)’ can be divided into 3 rows and 4 columns and each image can be placed in each position.  In the given example, number of frames =319 and the interval distance (Jump_ptr) is 27, then 319/27 gives 12. So the subplot is divided as 3 rows and 4 columns to allocate spacefor 12 images.

3. To read from a video file and write the frames to another video file

%To write frames to the video

Vptr = VideoReader('VIDE01.mp4');

Wptr = VideoWriter('VIDE02.mp4','MPEG-4');

Wptr.FrameRate=10;

open(Wptr);

for i=1:120

Img = readFrame(Vptr);

writeVideo(Wptr,Img);

end

close(Wptr);

EXPLANATION:

Create the video reader object using ‘VideoReader’ for ‘VIDEO1.mp4’

Create the video writer object using ‘VideoWriter’ for ‘VIDEO2.mp4’

Set the frame rate for the video to be written to a file.

Here, the frame rate 10 indicates,10 frames will be displayed per second in a video.

‘open’ command will open the video file to start the writing process. Instead of 319 frames from the original video(‘VIDEO1.MP4’), only 120 frames are written to the video file.So it is unnecessary to go through all the frames in the video. First read the frame from the input video file and write it to the output video file. After 120 frames are read from the input file and written to the output file, the output file is closed.

4. To read a video file and process the frames and write it to another video file

%To write frames to the video

%Create video Reader object

Vptr = VideoReader('VIDE01.mp4');

%Find number of frames

NFrames = round(Vptr.FrameRate*Vptr.Duration);

%Create Video Writer Object

Wptr = VideoWriter('VIDEO_NOISY.mp4','MPEG-4');

%Open the output video file

open(Wptr);

for i=1:NFrames

%Read from video file

Img = readFrame(Vptr);

%Add noise to the image

Img = imnoise(Img,'salt & pepper');

%write to video file

writeVideo(Wptr,Img);

end

%Close the output video file

close(Wptr);

EXPLANATION:

All the frames in the input video is processed and then written to an output file. Here, noise is added to each frame in the intermediate step and then written to the output video file. However, instead of addition of noise, the image can be enhanced or processed in the intermediate step.

EXAMPLE:

%EXAMPLE - VIDEO PROCESSING

%Set the frame rate

%Adjust the Image intensity

%Crop the Image

%Read 250 Frames

Vptr = VideoReader('VIDE01.mp4');

%Find number of frames

NFrames = round(Vptr.FrameRate*Vptr.Duration);

%Create Video Writer Object

Wptr = VideoWriter('VIDEO_Enhance.mp4','MPEG-4');

Wptr.FrameRate = 10;

%Open the output video file

open(Wptr);

for i=1:230

    %Read from video file

    Img = readFrame(Vptr);

   

    %Adjust the image intensity

    Img = imadjust(Img,[0 0 0; 0.7 0.7 0.5],[]);

   

    %Crop undesired portion

    Img = Img(1:end,251:end,:);

   

    %write to video file

    writeVideo(Wptr,Img);

end

%Close the output video file

close(Wptr);

EXPLANATION:

In this example, the frame rate is set to 10 and Instead of reading all the frames(319), 230 frames are read starting from the first frame. Each frame is enhanced and a portion of it is cropped as well. 

Color Slicing using HSV color space

Figure 1. 

   

          HSV color space can be used for assigning different colors to the foreground and background of the same image conveniently  in comparison to the equivalent RGB image. HSV color space consists of 3 components namely the Hue, the Saturation and the Value.


In MATLAB, HSV color space of an image is three dimensional matrix and each matrix represents each of the 3 component (Hue,Saturation,Value). Hue and saturation range between zero and one. While saturation defines colorfulness hue is specific to the color.

%MATLAB CODE:

A = imread('swimmer.jpg');

figure,imshow(A);title('Original Image');

Original Image

HSV = rgb2hsv(A);

H = HSV(:,:,1); %Hue

figure,imshow(H);colorbar;

Hue

H( H > mean2(H) ) = 1;

HSV(:,:,1) = H;

C = hsv2rgb(HSV);

figure,imshow(C);title('Hue Modified');

Hue Modified

EXPLANATION:

The original image is in RGB  format and it is converted to HSV color space using the MATLAB command ‘rgb2hsv’. The resultant is a three dimensional matrix with Hue, Saturation and Value components in each one of them. By comparing the original RGB image and the Hue component, we can understand that the blue color as high value comparing to other colors in the original image.

Hue is a color wheel, where the colors start from red, then move on to yellow, green, cyan, blue, magenta and ends up again in red.

In our example,the values above the average in the Hue matrix is made 1.(i.e. red). The background of the image is changed from blue to red.

If  the whole image including the swimmer needs to be changed to red then instead of finding the average or masking the image, assign zero or 1 to Hue matrix.

After the modification, use ‘hsv2rgb’ command to return back to RGB color space.

%MATLAB CODE:

HSV = rgb2hsv(A);

S = HSV(:,:,2); %Saturation

figure,imshow(S);colorbar;

Saturation

S(:,:)=0;

HSV(:,:,2) = S;

C = hsv2rgb(HSV);

figure,imshow(C);title('Saturation Modified');

Saturation Modified

 EXPLANATION:

 In this example, the saturation component is modified. The high values of Saturation illustrates that the regions are bright and colorful while the low values illustrates that they are dull and colorless.When saturation matrix is made zero, the colorfulness is completely lost. The above figure clearly shows the gray shade image that is obtained as a result of modifying the saturation matrix.

%MATLAB CODE:

HSV = rgb2hsv(A);

H = HSV(:,:,1); %Hue

S = HSV(:,:,2); %Saturation

H( H > mean2(H) ) = 0.42;

S( H < mean2(H) )=0;

S( H >= mean2(H) )=1;

HSV(:,:,2) = S;

HSV(:,:,1) = H;

C = hsv2rgb(HSV);

figure,imshow(C);title('Saturation Modified - Background');

EXPLANATION:

In this example, both the Hue and the Saturation matrices are modified simultaneously. The background color is changed from blue to green after changing the Hue matrix. The Saturation matrix is changed partially such that the background is not affected but the color on the swimmer is made gray.

%MATLAB CODE:

HSV = rgb2hsv(A);

H = HSV(:,:,1); %Hue

S = HSV(:,:,2); %Saturation

S( H < mean2(H) )=1;

S( H >= mean2(H) )=0;

HSV(:,:,2) = S;

HSV(:,:,1) = H;

C = hsv2rgb(HSV);

figure,imshow(C);title('Saturation Modified - Foreground');

EXPLANATION:

In this example, the background color is made shades of gray while the swimmer still retains the color. The masking is done based on the foreground and background on the saturation matrix and a value of zero is assigned to the background and one is assigned to foreground. From this example, it is evident that if the saturation matrix contains zero then the image in RGB color space will contain shades of gray whereas if the saturation matrix contains one then it will contain fully saturated more colorful image in the RGB color space.

The image(Figure.1) above shows the swimmer and different background colors by modifying the Hue matrix.