This week I experimented with new metrics, which I implemented.
I implemented 2 metrics based on color histogram and motion vectors. Since motion vectors already used in current version of the algorithm I focused on hostogram metric. Code for it can be found here. I used histogram crate for this implementation. I calculate histogram of the first plane of the frame (luma component) and compare it to the previous frame's histogram. I used 4 metrics to calculate differences between these histograms:
- The difference between mean values
- The difference between std values
- Taxicab distance
here p, q - histograms, n - amount of bins in each. - The square of the Euclidean distance
After that I substract current value from previous to make peaks more distinctive for the threshold.
Below you can find examples of these 4 distances in the same order on the same video. These pictures have scene changes (gray vertical lines) and final metric (blue line):
It can be seen that the metric with the least distinctive peaks is the one with STD difference.
Below you can find results for the first two distances:
- Mean values of histograms:
- STD values of histograms:
Below you can see the results for these metrics of BBC Planet Earth dataset and manually marked up open source videos:
| Metric | F score on BBC Planet Earth | F score on open source videos |
|---|---|---|
| #1(mean) | 0.8502 | 0.6532 |
| #2(std) | 0.6543 | 0.5951 |
| #3(Euclidian) | 0.7031 | 0.6002 |
| #4(Taxicab) | 0.7143 | 0.6231 |
The speed of the algorithm with this metric is ~0.87x speed of current version of the algorithm.
Summary:
The F score of new metrics is better than the current one.
New metrics are a bit slower that the current metric.
But since they use different characteristics of the frame (motion vectors and color histograms) in combination they could enhance each other and increase the final F score.
TO DO:
Precision recall curves for these metrics with different thresholds.
Correlation with previous metric. Would it be better to combine these metrics or use them separately?
Top comments (0)