DEV Community

loading...
Cover image for Videolitic - First Open source Project!

Videolitic - First Open source Project!

Michal Rogowski
Engineering Manager at YouGov.
・1 min read

Hey I was not posting for a while, because I’ve started working on open sourcing my side project called Videolitic. I’ve rewritten it to Swift + Combine + SwiftUI from Rx with UIKit so there is a lot of room for improvement - I’ve never done anything in SwiftUI and Combine before.

Videolitic. is an on-device video analysis framework. I am tracking each person from the video and detect his or her age, emotions and ethnicity, I am also creating transcript of the video. All results are combined and time-based.

This is how it works!
Example

My main goal is to create better open source models build on top of Vision which can detect:

  • Age
  • Gender
  • Ethnicity
  • Emotions

I wasn't able to OpenSource models which was trained on better datasets

The project is only library - I’ve added some UI just to present the results. There is a lot of Core ML + Vision + Create ML based on transfer learning.

I also integrated for the first time Travis-CI and it's GREAT!

I will create tutorial how to integrate Tavis-CI with the open source GitHub project and some articles with most interesting snippets!

Please checkout GitHub and contribute!

BTW! If you like my articles please follow me on Twitter Thanks!

Discussion (8)

Collapse
eavichay profile image
Avichay Eyal

This is a very interesting project and worth mentioning in a wider community.

My personal opinion regarding ethnicity classification is that it is nearly impossible to execute as many races have similar visual criteria but different origins and characteristics.

For example, Yemen people are confused by machine learning with Africans/African Americans etc.
Consider also the fact that within 2 or 3 decades most people in western economic countries will be a mix of races and looka cannot point to correct ethnicity classifications.

Gender is not yet an issue but this is also a changing subject.

I would extract ethnicity classification as this is impossible to predict.

Collapse
michalrogowski profile image
Michal Rogowski Author

Hey thank you for a kind word! I don't know how to mention it in a wider community but I will try to 👌

This is very interesting point of view, I always assumed that ethnicity is as simple as gender but probably as you wrote it's much harder to distinguish that with good confidence.

On the other hand maybe machine learning will be growing with us and more mix of races will be also visible in outputs of ML models - for example model has 60% certainty that someone is white and 40% that someone is asian because this person is a mix of races. As I wrote it's very interesting observation and may be no point in trying to create perfect model.

Thanks! :)

Collapse
layzee profile image
Lars Gyrup Brink Nielsen

Can we talk about how it's race and age predictions are so far from correct? 😄

Collapse
michalrogowski profile image
Michal Rogowski Author

Age model usually does a better job, but maybe it’s because she is wrinkle-free 🙂 about the ethnicity same answer I wrote below - dataset was bad :)

Collapse
supunkavinda profile image
Supun Kavinda

Wait, is Hillary Clinton black? 🤔

Collapse
michalrogowski profile image
Michal Rogowski Author

Yeah, for ethnicity I wasn’t able to use better model I’ve created to my non open source project because the dataset was created on different license:) Everything depends on datasets - I didn’t have time to create a script that can prepare a better one, but I was hoping maybe someone from the community has!

Collapse
kesprit profile image
kesprit

Great project 👏

Collapse
michalrogowski profile image
Michal Rogowski Author

Thank you!!