TensorFlow is a great platform for experimentation with learning models. About a year ago, I did some experiments with using the TensorFlow samples inside of the RICOH THETA 360 camera. A GitHub repo of the samples is available to you to experiment with if you have a RICOH THETA V or Z1. You can also run the standard demos on an Android phone or AVD.
The main problem using the TensorFlow demos with a 360 camera was detection of objects in an equirectangular image. In the example below, you can see that the 360 image viewed in equirectangular format distorts the faces.
This makes it challenging to access an image network such as YOLO (You Only Look Once).
Recently, developer Amine Amri, put out a project that introduced me to FDDB-360, a dataset derived from Face Detection Dataset and Benchmark FDDB.
FDDB-360 contains 17,052 fisheye-looking images and a total of 26,640 annotated faces.
The dataset is available from http://www.sfu.ca/~ibajic/#data (J. Fu, S. R. Alvar, I. V. Bajić, and R. G. Vaughan,“FDDB-360: Face detection in 360-degree fisheye images,” Proc. IEEE MIPR’19 ,San Jose, CA, Mar. 2019).
Working Without 360 Image DB
The TensorFlow demos work surprisingly well with 360 images without modification if the face is in the middle of the vertical height of the image.
You can even detect and blur faces close to a stitch line as long as the face is in the center of the vertical height.
When to use a 360 image DB of faces
Your recognition will decrease when the face is captured above or below the center of the vertical height. The higher or lower it is, the more distortion you will experience. For the best chance of face detection, you should use a 360 image DB.
Next Steps
If you want to join the discussion around using TensorFlow with the RICOH THETA and 360 images, please check out Tensorflow apk for THETA V - Speech, object detection, stylized filters.
Top comments (2)
Wow, the 360 degree datasets are "free for academic and non-commercial use." Nice find, thanks for posting.
Thanks. I would love to get more feedback from people using the dataset in their learning models. There's so much potential to use 360 cameras as a type of image capture sensor to grab the entire scene, not just from a front, sides, back view, but the entire scene