DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Cover image for Videolitic - First Open source Project!
Michal Rogowski
Michal Rogowski

Posted on

Videolitic - First Open source Project!

Hey I was not posting for a while, because I’ve started working on open sourcing my side project called Videolitic. I’ve rewritten it to Swift + Combine + SwiftUI from Rx with UIKit so there is a lot of room for improvement - I’ve never done anything in SwiftUI and Combine before.

Videolitic. is an on-device video analysis framework. I am tracking each person from the video and detect his or her age, emotions and ethnicity, I am also creating transcript of the video. All results are combined and time-based.

This is how it works!
Example

My main goal is to create better open source models build on top of Vision which can detect:

  • Age
  • Gender
  • Ethnicity
  • Emotions

I wasn't able to OpenSource models which was trained on better datasets

The project is only library - I’ve added some UI just to present the results. There is a lot of Core ML + Vision + Create ML based on transfer learning.

I also integrated for the first time Travis-CI and it's GREAT!

I will create tutorial how to integrate Tavis-CI with the open source GitHub project and some articles with most interesting snippets!

Please checkout GitHub and contribute!

BTW! If you like my articles please follow me on Twitter Thanks!

Top comments (8)

Collapse
 
eavichay profile image
Avichay Eyal

This is a very interesting project and worth mentioning in a wider community.

My personal opinion regarding ethnicity classification is that it is nearly impossible to execute as many races have similar visual criteria but different origins and characteristics.

For example, Yemen people are confused by machine learning with Africans/African Americans etc.
Consider also the fact that within 2 or 3 decades most people in western economic countries will be a mix of races and looka cannot point to correct ethnicity classifications.

Gender is not yet an issue but this is also a changing subject.

I would extract ethnicity classification as this is impossible to predict.

Collapse
 
michalrogowski profile image
Michal Rogowski Author

Hey thank you for a kind word! I don't know how to mention it in a wider community but I will try to πŸ‘Œ

This is very interesting point of view, I always assumed that ethnicity is as simple as gender but probably as you wrote it's much harder to distinguish that with good confidence.

On the other hand maybe machine learning will be growing with us and more mix of races will be also visible in outputs of ML models - for example model has 60% certainty that someone is white and 40% that someone is asian because this person is a mix of races. As I wrote it's very interesting observation and may be no point in trying to create perfect model.

Thanks! :)

Collapse
 
layzee profile image
Lars Gyrup Brink Nielsen

Can we talk about how it's race and age predictions are so far from correct? πŸ˜„

Collapse
 
michalrogowski profile image
Michal Rogowski Author

Age model usually does a better job, but maybe it’s because she is wrinkle-free πŸ™‚ about the ethnicity same answer I wrote below - dataset was bad :)

Collapse
 
supunkavinda profile image
Supun Kavinda

Wait, is Hillary Clinton black? πŸ€”

Collapse
 
michalrogowski profile image
Michal Rogowski Author

Yeah, for ethnicity I wasn’t able to use better model I’ve created to my non open source project because the dataset was created on different license:) Everything depends on datasets - I didn’t have time to create a script that can prepare a better one, but I was hoping maybe someone from the community has!

Collapse
 
kesprit profile image
kesprit

Great project πŸ‘

Collapse
 
michalrogowski profile image
Michal Rogowski Author

Thank you!!

Find what you were looking for? Join hundreds of thousands of developers on DEV so you can:

Β 
🌚 Enable dark mode
πŸ”  Change your default font
πŸ“š Adjust your experience level to see more relevant content