Most career advice relies on survey reports or recruiter lists that treat all skill mentions equally. A casual mention of Python in an introductory paragraph gets counted the same as five years of required experience in the qualifications section.
To get clearer data for my own job search, I built an open source tool called StackCheck.
How it works:
- Pulls live postings matching your search criteria directly from HiringCafe.
- Separates job descriptions into sections and weights hard qualifications higher than general overview text.
- Normalizes hourly, monthly, and international compensation disclosures into standardized annual USD brackets with 25th, median, and 75th percentiles.
- Computes co-occurrence matrices showing which technologies companies actually demand together (such as Python with Snowflake versus Python with Django).
- Runs completely on your own machine. Data is stored in a local SQLite file with zero cloud accounts, telemetry, or mandatory API keys.
It includes both a command line interface and a local browser dashboard for inspecting postings, sorting by salary, and filtering by workplace type (remote, hybrid, onsite).
Source code and prebuilt binaries: https://github.com/haydermuhib/StackCheck
Web documentation: https://haydermuhib.github.io/StackCheck/
Hope it helps anyone researching market demand for their specific stack.
Top comments (1)
Thực tế parse JD live cho insights này khá thú vị — survey data thường bị bias nặng vì dev tự report stack mình biết chứ không phải stack thực tế đang dùng ở job hiện tại.
Một vài thách thức mình thấy khi làm việc với data dạng này:
Deduplication across boards — cùng một role đăng trên LinkedIn, Indeed, company career page thường có description hơi khác nhau. Bạn handle như thế nào? Fuzzy match trên title + company + location?
Salary parsing — ranges viết kiểu "100k-150k", "up to 200k", "negotiable", hoặc chỉ có "competitive". Normalize về percentile P25/P50/P75 cần logic khá phức tạp, đặc biệt khi currency/location mixed.
Tech stack extraction — "React" vs "React.js" vs "ReactJS" vs "React (hooks, context)" — bạn dùng NER hay pattern matching? Mình từng thử spaCy custom entity ruler cho stack detection, recall ổn nhưng precision bị noise bởi mention trong section "nice to have" vs "required".
Seniority inference — title có "Senior" nhưng YOE requirement 3 năm, hoặc ngược lại. Có weight years of experience từ JD không?
Curious xem approach của bạn cho phần normalization/cleaning pipeline. Data quality ở layer này quyết định toàn bộ signal downstream — found it via LabAgent, site: labagent .tech