Welcome Guest ( Log In | Register )

Outline · [ Standard ] · Linear+

 Evaluate similarity between patches of image, algorithm, method or approach

views
     
TSweisinx7
post Mar 11 2020, 10:10 AM, updated 5y ago

Getting Started
**
Junior Member
212 posts

Joined: Oct 2019
Hi,

I'm trying to match patches of image so would like to seek for any further inputs here.

Right now , what in my mind is:

1. Using edges (find distance of the binary edge image, but this only solve shape problem)
2. Intensity (Maybe colour histogram, but not sure on the evaluation metric)
3. Apply Filters (to get some filtered response, and match from there)

Few other problem i can see but not sure how to go around it :

1. Gamma correction ( is there a way to output an image with consistent intensity?)
2. Evaluation metric ( i can see there are a few, but not sure which is better, for example, euclidean distance, hamming distance, mahalanobis distance and etc)

Any input is appreciated.

Thanks


TSweisinx7
post Mar 11 2020, 11:23 AM

Getting Started
**
Junior Member
212 posts

Joined: Oct 2019
QUOTE(Palindromes @ Mar 11 2020, 11:01 AM)
This is an interesting topic. May I know what do you mean by patches of images?

Or perhaps you can give me two sample image similar to each other? I want to test... biggrin.gif
*
Patches of image means a small portion of the image, or a cropped section. I want to find the similarity index for 2 images, but i do it with smaller scale, separate the image to patches of image and then compare each of it.

For example:

Just compare the patches of the red square section:

user posted image
Palindrome
post Mar 11 2020, 11:58 AM

New Member
*
Validating
30 posts

Joined: Feb 2020
Hi TS, without referring to your latest reply, I have attempted to Bitmap Compare method below.

For example: The two following image will have Similarity = 91.16% according to my C# code:
user posted image
user posted image

It calculates similarity based on threshold of luminance. (The formula of luminance is NOT mine, however)

user posted image
user posted image

When refer Bmp1 and Bmp2 to the same image, the similarity is always 100%.

Now I am not sure if this applies to your patches of image, in your latest reply. tongue.gif

CODE
using System;
using System.Drawing;

namespace CMPBMP
{
   class Program
   {
       static void Main(string[] args)
       {
           Bitmap Bmp1 = new Bitmap("C:\\watermelon_1.jfif");
           Bitmap Bmp2 = new Bitmap("C:\\watermelon_2.jfif");

           var Threshold = 127; // Any value between 0 ..255
           double Multiplier = 600; // New size of bitmap width          
           double Scale = (double)(Multiplier / (double)Bmp1.Width);
           int NewHeight1 = (int)(Bmp1.Height * Scale);
           int NewWidth1 = (int)(Bmp1.Width * Scale);
           byte[] Bitmap1 = new byte[NewWidth1 * NewHeight1];

           for (var y = 0; y < NewHeight1; y++)
           {
               for (var x = 0; x < NewWidth1; x++)
               {
                   var _x = (int)(x / Scale);
                   var _y = (int)(y / Scale);
                   var Color = Bmp1.GetPixel(_x, _y);
                   var Luminance = (int)(Color.R * 0.3 + Color.G * 0.59 + Color.B * 0.11);
                   if (Luminance < Threshold)
                       Bitmap1[_x * _y] = 255;
                   else
                       Bitmap1[_x * _y] = 0;
               }

           }

           Scale = (double)(Multiplier / (double)Bmp2.Width);
           int NewHeight2 = (int)(Bmp2.Height * Scale);
           int NewWidth2 = (int)(Bmp2.Width * Scale);
           byte[] Bitmap2 = new byte[NewWidth2 * NewHeight2];
           for (var y = 0; y < NewHeight2; y++)
           {
               for (var x = 0; x < NewWidth2; x++)
               {
                   var _x = (int)(x / Scale);
                   var _y = (int)(y / Scale);
                   var Color = Bmp2.GetPixel(_x, _y);
                   var Luminance = (int)(Color.R * 0.3 + Color.G * 0.59 + Color.B * 0.11);
                   if (Luminance < Threshold)
                       Bitmap2[_x * _y] = 255;
                   else
                       Bitmap2[_x * _y] = 0;
               }
           }

           int NewHeight = NewHeight1 > NewHeight2 ? NewHeight2 : NewHeight1;
           double Similarity = 0;

           for (int y = 0; y < NewHeight; y++)
               for (int x = 0; x < NewWidth1; x++)
               {
                   if (Bitmap1[y * x] == Bitmap2[y * x])
                       Similarity++;
               }

           Console.WriteLine("Similarity = " + ((Similarity / (NewHeight * NewWidth1)) * 100).ToString("F2") + "%"); //Similarity = 91.16%
           Console.ReadLine();
       }
   }
}



Palindrome
post Mar 11 2020, 12:02 PM

New Member
*
Validating
30 posts

Joined: Feb 2020
It is best if have same size of image, since I am too hurried to have the Bitmap Compare method done, the code will not work well for image with different size (particularly different height of image).....Just prototype...hah


Palindrome
post Mar 11 2020, 12:23 PM

New Member
*
Validating
30 posts

Joined: Feb 2020
QUOTE(weisinx7 @ Mar 11 2020, 11:23 AM)
Patches of image means a small portion of the image, or a cropped section. I want to find the similarity index for 2 images, but i do it with smaller scale, separate the image to patches of image and then compare each of it.

For example:

Just compare the patches of the red square section:

user posted image
*
P1:
Attached Image

P2:
Attached Image

P3:
Attached Image

P4:
Attached Image

Just some numbers generated by Bitmap Compare method.

Threshold=127
P1 & P2: Similarity = 97.47%
P1 & P3: Similarity = 97.64%
P3 & P4: Similarity = 98.34%

Threshold=48
P1 & P2: Similarity = 99.80%
P1 & P3: Similarity = 99.71%
P3 & P4: Similarity = 99.82%

I think this code is only suitable for detecting image with different object and position only.

cassian948
post Mar 11 2020, 12:27 PM

Enthusiast
*****
Junior Member
890 posts

Joined: Jan 2017
From: Kuala Lumpur
I've read a thesis about this, but can't recalled the method they used.
TSweisinx7
post Mar 11 2020, 12:48 PM

Getting Started
**
Junior Member
212 posts

Joined: Oct 2019
Thanks for the effort, but actually i'm looking for a more robust solution, which enable me to detect some irregular items within the images. Hence, i'm thinking to fuse several approaches, such as edges + intensity or any other method that i didn't aware of. In your case, bitmap comparison would be the comparison in intensity level, maybe you can try with more images, then you would see the limitation
TSweisinx7
post Mar 11 2020, 12:55 PM

Getting Started
**
Junior Member
212 posts

Joined: Oct 2019
maybe you can try this:


There are 5 images.
first 2 images should not be similar with the other 3 images. The other 3 images should be similar



user posted image
user posted image
user posted image
user posted image
user posted image
TSweisinx7
post Mar 11 2020, 02:54 PM

Getting Started
**
Junior Member
212 posts

Joined: Oct 2019
QUOTE(Palindromes @ Mar 11 2020, 02:29 PM)
Yes, using my code above will resulting in over 99% similarities in all comparison.  confused.gif

After I peek into the bitmap representation of the 5 images, I tweaked it a little bit, so that it uses inverted image, and the result is in number of dots matches. As below:
As what you said, the latter three images are most identical to each other. While second image is closer to the latter three than the first image is closer to the latter three.

Ok, thank you for giving me the opportunity to practise bitmap handling. However, my skill in image processing is limited. 

I hope you can share what your finding in the future on here too. Keep us posted.
*
Thanks for trying out, do you mind to share your code?

Right now my constraint is to find the suitable evaluation metric, as i can find the edges, filtered value and etc, but to give a score on the similarity, that is still a problem, since the score need to be work on all cases
Palindroms P
post Mar 11 2020, 03:56 PM

New Member
*
Probation
3 posts

Joined: Mar 2020
I am sorry, my previous IDs have running of daily post quota....

QUOTE(weisinx7 @ Mar 11 2020, 02:54 PM)
Thanks for trying out, do you mind to share your code?

Right now my constraint is to find the suitable evaluation metric, as i can find the edges, filtered value and etc, but to give a score on the similarity, that is still a problem, since the score need to be work on all cases
*
Yeah, I am having difficulty calculate the percentage of similarity also (out of number of dots matched).

Bmp1
Attached Image

Bmp2
Attached Image

Bmp3
Attached Image

Bmp4
Attached Image

Bmp5
Attached Image

Please note that the code doesn't work well with image with different height (different width is okay, as it will resize it to 600 pixels)

Please do note the following
CODE
                  if (((Bitmap1Inverted[y * x] == Bitmap2Inverted[y * x])) && (Bitmap1Inverted[y * x] ==255))
                       SimilarityInverted++;


The number of dots matched will calculate those black dots if found in two images.

You can play around with threshold to higher value or lower value....... I hope it can be somewhat useful for you, though bitmap comparison is somewhat lower-class.... tongue.gif


CODE
using System;
using System.Drawing;

namespace CMPBMP
{
   class Program
   {
       static void Main(string[] args)
       {
           Bitmap Bmp1 = new Bitmap("C:\\bmp2.png");
           Bitmap Bmp2 = new Bitmap("C:\\bmp5.png");


           var Threshold = 100; // Any value between 0 ..255
           double Multiplier = 600; // New size of bitmap width          
           double Scale = (double)(Multiplier / (double)Bmp1.Width);
           int NewHeight1 = (int)(Bmp1.Height * Scale);
           int NewWidth1 = (int)(Bmp1.Width * Scale);
           byte[] Bitmap1 = new byte[NewWidth1 * NewHeight1];
           byte[] Bitmap1Inverted = new byte[NewWidth1 * NewHeight1];
           int NumberOfInvertedDots = 0;

           for (var y = 0; y < NewHeight1; y++)
           {
               for (var x = 0; x < NewWidth1; x++)
               {
                   var _x = (int)(x / Scale);
                   var _y = (int)(y / Scale);
                   var Color = Bmp1.GetPixel(_x, _y);
                   var Luminance = (byte)(Color.R * 0.3 + Color.G * 0.59 + Color.B * 0.11);
                   if (Luminance < Threshold)
                   {
                      Bitmap1Inverted[_x * _y] = 0;
                       
                       Bitmap1[_x * _y] = 255;
                   }
                   else
                   {
                      Bitmap1Inverted[_x * _y] = 255;
                       NumberOfInvertedDots++;
                       Bitmap1[_x * _y] = 0;
                   };

                   //Bitmap1Inverted[_x * _y] = Luminance;

               }

           }

           Scale = (double)(Multiplier / (double)Bmp2.Width);
           int NewHeight2 = (int)(Bmp2.Height * Scale);
           int NewWidth2 = (int)(Bmp2.Width * Scale);
           byte[] Bitmap2 = new byte[NewWidth2 * NewHeight2];
           byte[] Bitmap2Inverted = new byte[NewWidth2 * NewHeight2];
           for (var y = 0; y < NewHeight2; y++)
           {
               for (var x = 0; x < NewWidth2; x++)
               {
                   var _x = (int)(x / Scale);
                   var _y = (int)(y / Scale);
                   var Color = Bmp2.GetPixel(_x, _y);
                   var Luminance = (byte)(Color.R * 0.3 + Color.G * 0.59 + Color.B * 0.11);
                   if (Luminance < Threshold)
                   {
                       Bitmap2Inverted[_x * _y] = 0;
                       Bitmap2[_x * _y] = 255;
                   }
                   else
                   {
                       Bitmap2Inverted[_x * _y] = 255;
                       Bitmap2[_x * _y] = 0;
                   };
                   //Bitmap2Inverted[_x * _y] = Luminance;
               }
           }

           int NewHeight = NewHeight1 > NewHeight2 ? NewHeight2 : NewHeight1;
           double Similarity = 0;
           double SimilarityInverted = 0;

           for (int y = 0; y < NewHeight; y++)
               for (int x = 0; x < NewWidth1; x++)
               {
                   if (Bitmap1[y * x] == Bitmap2[y * x])
                       Similarity++;
                   if (((Bitmap1Inverted[y * x] == Bitmap2Inverted[y * x])) && (Bitmap1Inverted[y * x] ==255))
                       SimilarityInverted++;
               }

           Console.WriteLine("Similarity (Bitmap) = " + ((Similarity / (NewHeight * NewWidth1)) * 100).ToString("F2") + "%");
           Console.WriteLine("Number of dots matched (Inverted Bitmap) = " + SimilarityInverted.ToString("F2"));
           Console.ReadLine();
       }
   }
}



TSweisinx7
post Mar 11 2020, 04:00 PM

Getting Started
**
Junior Member
212 posts

Joined: Oct 2019
QUOTE(Palindroms @ Mar 11 2020, 03:56 PM)
I am sorry, my previous IDs have running of daily post quota....
Yeah, I am having difficulty calculate the percentage of similarity also (out of number of dots matched).

Bmp1
Attached Image

Bmp2
Attached Image

Bmp3
Attached Image

Bmp4
Attached Image

Bmp5
Attached Image

Please note that the code doesn't work well with image with different height (different width is okay, as it will resize it to 600 pixels)

Please do note the following
CODE
                  if (((Bitmap1Inverted[y * x] == Bitmap2Inverted[y * x])) && (Bitmap1Inverted[y * x] ==255))
                       SimilarityInverted++;


The number of dots matched will calculate those black dots if found in two images.

You can play around with threshold to higher value or lower value....... I hope it can be somewhat useful for you, though bitmap comparison is somewhat lower-class....  tongue.gif
CODE
using System;
using System.Drawing;

namespace CMPBMP
{
   class Program
   {
       static void Main(string[] args)
       {
           Bitmap Bmp1 = new Bitmap("C:\\bmp2.png");
           Bitmap Bmp2 = new Bitmap("C:\\bmp5.png");
           var Threshold = 100; // Any value between 0 ..255
           double Multiplier = 600; // New size of bitmap width          
           double Scale = (double)(Multiplier / (double)Bmp1.Width);
           int NewHeight1 = (int)(Bmp1.Height * Scale);
           int NewWidth1 = (int)(Bmp1.Width * Scale);
           byte[] Bitmap1 = new byte[NewWidth1 * NewHeight1];
           byte[] Bitmap1Inverted = new byte[NewWidth1 * NewHeight1];
           int NumberOfInvertedDots = 0;

           for (var y = 0; y < NewHeight1; y++)
           {
               for (var x = 0; x < NewWidth1; x++)
               {
                   var _x = (int)(x / Scale);
                   var _y = (int)(y / Scale);
                   var Color = Bmp1.GetPixel(_x, _y);
                   var Luminance = (byte)(Color.R * 0.3 + Color.G * 0.59 + Color.B * 0.11);
                   if (Luminance < Threshold)
                   {
                      Bitmap1Inverted[_x * _y] = 0;
                       
                       Bitmap1[_x * _y] = 255;
                   }
                   else
                   {
                      Bitmap1Inverted[_x * _y] = 255;
                       NumberOfInvertedDots++;
                       Bitmap1[_x * _y] = 0;
                   };

                   //Bitmap1Inverted[_x * _y] = Luminance;

               }

           }

           Scale = (double)(Multiplier / (double)Bmp2.Width);
           int NewHeight2 = (int)(Bmp2.Height * Scale);
           int NewWidth2 = (int)(Bmp2.Width * Scale);
           byte[] Bitmap2 = new byte[NewWidth2 * NewHeight2];
           byte[] Bitmap2Inverted = new byte[NewWidth2 * NewHeight2];
           for (var y = 0; y < NewHeight2; y++)
           {
               for (var x = 0; x < NewWidth2; x++)
               {
                   var _x = (int)(x / Scale);
                   var _y = (int)(y / Scale);
                   var Color = Bmp2.GetPixel(_x, _y);
                   var Luminance = (byte)(Color.R * 0.3 + Color.G * 0.59 + Color.B * 0.11);
                   if (Luminance < Threshold)
                   {
                       Bitmap2Inverted[_x * _y] = 0;
                       Bitmap2[_x * _y] = 255;
                   }
                   else
                   {
                       Bitmap2Inverted[_x * _y] = 255;
                       Bitmap2[_x * _y] = 0;
                   };
                   //Bitmap2Inverted[_x * _y] = Luminance;
               }
           }

           int NewHeight = NewHeight1 > NewHeight2 ? NewHeight2 : NewHeight1;
           double Similarity = 0;
           double SimilarityInverted = 0;

           for (int y = 0; y < NewHeight; y++)
               for (int x = 0; x < NewWidth1; x++)
               {
                   if (Bitmap1[y * x] == Bitmap2[y * x])
                       Similarity++;
                   if (((Bitmap1Inverted[y * x] == Bitmap2Inverted[y * x])) && (Bitmap1Inverted[y * x] ==255))
                       SimilarityInverted++;
               }

           Console.WriteLine("Similarity (Bitmap) = " + ((Similarity / (NewHeight * NewWidth1)) * 100).ToString("F2") + "%");
           Console.WriteLine("Number of dots matched (Inverted Bitmap) = " + SimilarityInverted.ToString("F2"));
           Console.ReadLine();
       }
   }
}

*
Thanks, but if we inspect visually, wouldn't be img 2 and img 4 looks similar for the binarized image?

Palindroms P
post Mar 11 2020, 04:25 PM

New Member
*
Probation
3 posts

Joined: Mar 2020

QUOTE(weisinx7 @ Mar 11 2020, 04:00 PM)

Thanks, but if we inspect visually, wouldn't be img 2 and img 4 looks similar for the binarized image?
*



Yes, at least it is closer than bmp3 vs bmp5,

I believe more talented coder will pour in idea on your similarity evaluation problem.... icon_question.gif

https://pictr.com/images/2020/03/11/5X766q.md.png
TSweisinx7
post Mar 11 2020, 04:57 PM

Getting Started
**
Junior Member
212 posts

Joined: Oct 2019
QUOTE(Palindroms @ Mar 11 2020, 04:25 PM)
Yes, at least it is closer than bmp3 vs bmp5,
I believe more talented coder will pour in idea on your similarity evaluation problem....  icon_question.gif
<a href='https://pictr.com/images/2020/03/11/5X766q.md.png' target='_blank'>https://pictr.com/images/2020/03/11/5X766q.md.png </a>
*
right now i'm exploring on the ssim (structural similarity index )and i think the results are quite good:

http://www.programmersought.com/article/3454534312/

Perhaps you would be interested as well

 

Change to:
| Lo-Fi Version
0.0142sec    0.29    6 queries    GZIP Disabled
Time is now: 29th March 2024 - 10:33 PM