🏁 My Capstone Project
My final project for MS degree in data science is to help our client, a heavy machinery online auction firm to verify their long-held hypothesis that image features do influence the product attractiveness and build an accurate and interpretable auction price prediction model using image features, essential data (such as age, usage and recent market conditions), text features and MTURK annotations for images.
- Note: We use images of skid steer, a small piece of construction equipment primarily used for digging, in our project because they are auctioned most frequently.
Drawing upon published research about image processing techniques (image colorfulness), sentiment analysis (VADER) and attention network for prediction interpretation (Class Activation Map (CAM), Guided Saliency Map), my teammate and I ultimately proposed a multi-layer price prediction model that concatenates a CNN model, essential data, text and other features to predict eventual sales price of a skid steer. We benchmarked performance of different hyper-parameters and optimize the final prediction with mean absolute error of $2541.85 (19.06%). We conducted quantitative feature importance analysis of all the interpretable features. In addition, qualitative network attention algorithms are implemented by color coding important pixels.
In addition to CNN produced image features, we also collected annotations for images using Mechanical Turk (MTURK) to extract interpretable image features. We worked with our client to design 13 questions that could potentially extract as much information from the images as possible (including: background weather, rust extent, brightness of color, dirt level, etc). We also explored the effects of adding MTURK data in the model to investigate if features in images are important.
🔗 Demo Link
A Model Interpretation App For Price prediction
- Note: This is only a small demo that only samples and saves some model output images. Please refer to the white paper for the algorithm.
🔗 Link to Code
Demo Code
White Paper (if you're interested in the details of data collection, algorithm usage and analysis.)
⚒ How I built it
We used Streamlit to build an interactive visualization demo app for our client to explore model's price prediction and how different features contribute to the prediction. For the random forest model, we use Shapely Value to give a local interpretation for each prediction whereas in neural network model, we sampled output images from model and provide our client qualitative interpretation of where the CNN model focuses for price prediction.
💡 Additional Thoughts (Soft Skills)
This is a year long project where my teammate and I corporated with our client to deliver something "impactful & useful" in a real world scenario. We didn't have any project management experience before, but coming out of this project, we picked up really good practices by updating our clients regularly, using Trello to create ticketing system, seeking wider community for help by cold-emailing researchers and invaluable skills to break down a gigantic task into small, doable chunks.
Top comments (0)