The Eagerly Awaited Needs Assessment Results

Hello, Everyone.  

I’m excited to be able to share the Needs Assessment Survey results with you.  It took us a bit longer than expected to release them, but the psychometric firm that analyzed the data was meticulous and thorough in their review.  It was important to us that the profession as a whole knows that it was not just NCBTMB interpreting the results for itself.  We received bids from three firms and, after review and discussions at the National Organization for Competency Assurance (NOCA) Annual Conference (NOCA is now ICE – the Institute for Credentialing Excellence), we selected Mountain Measurement, Inc.  They were sent all of the survey raw data (the results and responses) and then they compiled, reviewed and analyzed the data to create the Needs Assessment Survey Report. 

The data was reported using descriptive statistics and, where appropriate, analyzed using chi square techniques.  By using this report format, we were able to prevent the results from becoming unwieldy while protecting the integrity of the measureable data.  It would have been impractical to release the raw survey data because of the survey methodologies used (including branching and open-ended responses.)  The data reflects information gathered from everyone – regardless of whether they are "shiny, happy" people or disgruntled individuals.

To me, one of the most surprising things was the fact that almost 25% of the respondents were not certified.  I am grateful to everyone who helped to spread the word of this survey and who helped us get responses from across the profession.  I think their efforts really helped us to get an overall feel for the thoughts of all practitioners, not just those who hold national certification.    

I also found it quite interesting that respondents were split 50/50 on the question of whether a hands-on assessment is necessary.  Hands-on assessments are tricky because they are very subjective.  They are also expensive and difficult to administer.  We have been exploring different forms of computer-based assessments that may be closer to a hands-on methodology than the traditional text-based multiple choice questions, so perhaps they will present an option for the advanced certification assessment. 

Now that these results are in, the Eligibility Subgroup can finish its work on the eligibility criteria.  Once that work is completed, the work on the next survey – the Job Task Analysis Survey – will get underway.  Please keep your eyes open for the JTA announcement and watch the arrow progress up the stairs on the Exam Development Step-By-Step document.

 

Liz