Your theory around the survey not being very beginner friendly I think may be a good path to go down. I would even say that it's not very intermediate friendly either. Even as someone that has been around the industry for a while and has a good amount of knowledge around existing features and frameworks, I find the survey somewhat intimidating. I can't imagine how much worse it is for someone less experienced.
Generally, the first section about the features that you know or are aware of feels like a big test of how good of a programmer you are, so it's pretty disheartening to come away from that very first section feeling like you don't know very much. And every year it adds new things and replaces older things, so even if you came to the survey again the year after now knowing more about that topic that you didn't know about before, you're greeted with more new things where you once again get to experience a feeling of imposter syndrome. It's only after the survey results come out where you get to feel like maybe you're not the only one who doesn't know about X feature.
Even though I've taken the survey throughout the years, I find myself wondering why I keep taking it and what the purpose even is for some of the questions. I would say I usually don't feel great coming out of that survey.
Perhaps some things to consider:
How much of the survey is actually measuring the current state of JS/CSS and how much is measuring the future state of JS/CSS?
How could you frame things to make the survey more welcoming to those who don't keep up with every new thing that comes out?
How much of the survey is actually measuring the current state of JS/CSS and how much is measuring the future state of JS/CSS?
This is key, the goal of the survey is anticipating future trends, which is why if we do our job right you shouldn't know about most things in the survey. In fact, once it becomes clear that most people already know/use a feature and the trend is now pretty stable (things like Flexbox or CSS Grid for example) we remove it from the survey altogether.
How could you frame things to make the survey more welcoming to those who don't keep up with every new thing that comes out?
Great question. I think the answer might be bringing more value to people taking the survey even if they don't know the items in it. Maybe find a way to make the survey more fun or more educational?
Great question. I think the answer might be bringing more value to people taking the survey even if they don't know the items in it. Maybe find a way to make the survey more fun or more educational?
I've found out about more than one cool upcoming feature by seeing it in one of the surveys and looking it up.
Adding some feedback for things respondents don't know about, like a "Find out what this is" popup with a brief explanation and a link might be a way to counteract this?
Anyway, this brings me to another question: Is there any data on respondents not finishing the survey? My suspicion is that a fair amount of respondents might stop the survey halfway-through as they realise how much they don't know, rather than just not participating in the first place.
What would be a way to address this, other than having microsoft paperclip go tok tok tok "by the way it's fine if you don't know most of this"? My rough ideas would be:
Communicating clearly that these are upcoming features and not things someone necessarily needs to know about (yet)
Phrasing the question in a non-accusatory way, not framing it as a skill issue on their part
Just generally focusing more on what people do know instead of long lists of things they never heard about.
This is key, the goal of the survey is anticipating future trends, which is why if we do our job right you shouldn't know about most things in the survey. In fact, once it becomes clear that most people already know/use a feature and the trend is now pretty stable (things like Flexbox or CSS Grid for example) we remove it from the survey altogether.
Interesting! I think it would be helpful to make this more explicit. I also wonder then if it's worth including a little blurb about each one and see, when someone hasn't used something or know what it is, if they think they'd find it useful and/or want to learn more about it? Although I don't know if that changes the purpose of the results.
Great question. I think the answer might be bringing more value to people taking the survey even if they don't know the items in it. Maybe find a way to make the survey more fun or more educational?
That sounds like an interesting idea! Not sure how complicated this would be, but it would be cool to have a custom resource at the end that's based on responses where someone says they haven't used X is but are interested.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Your theory around the survey not being very beginner friendly I think may be a good path to go down. I would even say that it's not very intermediate friendly either. Even as someone that has been around the industry for a while and has a good amount of knowledge around existing features and frameworks, I find the survey somewhat intimidating. I can't imagine how much worse it is for someone less experienced.
Generally, the first section about the features that you know or are aware of feels like a big test of how good of a programmer you are, so it's pretty disheartening to come away from that very first section feeling like you don't know very much. And every year it adds new things and replaces older things, so even if you came to the survey again the year after now knowing more about that topic that you didn't know about before, you're greeted with more new things where you once again get to experience a feeling of imposter syndrome. It's only after the survey results come out where you get to feel like maybe you're not the only one who doesn't know about X feature.
Even though I've taken the survey throughout the years, I find myself wondering why I keep taking it and what the purpose even is for some of the questions. I would say I usually don't feel great coming out of that survey.
Perhaps some things to consider:
I think you make some great points!
This is key, the goal of the survey is anticipating future trends, which is why if we do our job right you shouldn't know about most things in the survey. In fact, once it becomes clear that most people already know/use a feature and the trend is now pretty stable (things like Flexbox or CSS Grid for example) we remove it from the survey altogether.
Great question. I think the answer might be bringing more value to people taking the survey even if they don't know the items in it. Maybe find a way to make the survey more fun or more educational?
I've found out about more than one cool upcoming feature by seeing it in one of the surveys and looking it up.
Adding some feedback for things respondents don't know about, like a "Find out what this is" popup with a brief explanation and a link might be a way to counteract this?
Anyway, this brings me to another question: Is there any data on respondents not finishing the survey? My suspicion is that a fair amount of respondents might stop the survey halfway-through as they realise how much they don't know, rather than just not participating in the first place.
What would be a way to address this, other than having microsoft paperclip go tok tok tok "by the way it's fine if you don't know most of this"? My rough ideas would be:
Great feedback, thanks!
Interesting! I think it would be helpful to make this more explicit. I also wonder then if it's worth including a little blurb about each one and see, when someone hasn't used something or know what it is, if they think they'd find it useful and/or want to learn more about it? Although I don't know if that changes the purpose of the results.
That sounds like an interesting idea! Not sure how complicated this would be, but it would be cool to have a custom resource at the end that's based on responses where someone says they haven't used X is but are interested.