I taught a postgraduate class on image segmentation for urban planning

This was back in 2021

I have to start off by saying many thanks to my good friend Siddharth Khakhar for this opportunity and under the guidance of Associate Professor Alain J.F. Chiaradia.

The course MUDP2020 as part of the HKU Master of Urban Design students were taught how to use a range of data collection/generation and analysis. Some are well known, and some are relatively novel in the programme, like social data sentiment analysis and image segmentation analysis. They might be crude to date – yet they will get better – we got to start anywhere.

There were 4 main themes to this course, and I taught the theme "Street view images in urban research" to a group of students in this course. This course was not about teaching coding, but to introduce new tools that could be used in the field for the future of city planning.

My work

What are we trying to achieve?
Google street view images is a powerful tool for urban designers. They use it daily to analyze new prospects, carry research on existing cities, etc. My goal was to build a pipeline on analyzing an area using image segmentation. This should aid students on complementing this with other data to answer questions like "Is there enough greenery in the pathway?", "Are there any natural stressors present in this area leading to health issues?" (more on this later).

Do it all python script 🔗
Python script that does a batched analysis.

Steps:

1. Download the 360 panoramic images of an area. (Students were given access to SVD360 Pro to download the area in few steps)
2. Load the images onto Google Drive
3. Run the script

Actions:

1. Crop 360 panorama images to the different angles, so that analysis can be done on one particular view.

Cropping is required on panoramic photos. For example, left view could be used for sidewalk analysis

1. Run segmentation on the images on a pretrained model 'mobilenetv2_coco_cityscapes_trainfine'

Segmentation overlay as seen by the model

1. Produce report on excel

• file_path
• data_time_original
• google_street_view_image_id
• gps_info
• altitude
• segmentation categories (road, sidewalk, building, wall, fence, pole, traffic light, terrain, vegetation, sky, person, vehicles)

Challenges

Students had little/no experience with coding/python. The class has to be conducted remotely for 3 out of the 5 months of the semester due to the lockdown for COVID in HK. This was also my first time teaching and therefor I was doubtful in each step if the students understood what I was saying. Also, this was the first time the course was being taught.

Student takeaways

Google Colab and Google Drive based script was very well welcomed by the students as it didn't require them to setup the python. The course review and instructor review was above average and received generous feedback.

Out of all the student researches what caught my eye was as argument on using image segmentation based techniques to further explore on the research done by Honold et. al. on Multiple environmental burdens and neighborhood-related health of city residents. This group of students devised a research framework that uses image segmentation to get data for building quality, sky view, vegetation, complemented with other data (traffic noise, questionnaires) to identify natural stressors in a given community.