How to build a certification exam blueprint from a job task analysis

A certification exam blueprint is the plan that says how many scored items each content domain gets, at what cognitive level, on every form of the exam. You build it from a job task analysis: experts list the tasks the job involves, practitioners rate each task for importance and frequency, and those ratings become the domain weights. Item counts, bank size and forms all follow from that.

In LMS Advisor, the approved blueprint maps straight onto the question bank: one pool per domain, items tagged by objective and set to a difficulty level, and a Test Center exam that candidates sit through their own portal, proctored if the credential needs it. The method below is platform-neutral, so it works wherever your exam lives.

Most guidance on this is written by psychometric consultancies for other psychometricians. This walkthrough is for the program manager who has to get one approved. We've built and rebuilt exam banks inside LMS platforms for years, and the same mistakes show up every time: weights picked in a meeting, objectives nobody can write three good items for, and a bank that's one form deep.

What is a certification exam blueprint, and why does it come first?

An exam blueprint (also called the test specifications or content outline) is a table. Rows are domains and objectives. Columns are the share of the exam each one gets, the number of scored items, and the cognitive level those items target. Some add item types or the job tasks each objective traces to.

It comes first because every later decision depends on it. Item writers need it to know what to write. Your psychometrician needs it to assemble equivalent forms. Candidates need a public version to know what to study. And if you ever seek accreditation, you'll need to show the link between the job and the exam. The 2021 NCCA Standards for the Accreditation of Certification Programs say in Standard 15 that the plan for weighting sections of an exam must be based on a job analysis, and Standard 6 expects programs to publish a description of the exam, including its content domains and weightings.

A blueprint built from opinion is hard to defend when a failed candidate asks why the exam covered what it did. One built from practitioner survey data answers with evidence.

Step 1: how do you run a job task analysis for certification?

A job task analysis (JTA), sometimes called a job analysis or practice analysis, is a structured study of what competent people in a role actually do. Assessment Systems' overview of a JTA study lays out the usual sequence, and it matches what we see in practice:

1. Convene and train an SME panel. Pick subject matter experts who do the job now, not only managers or trainers. Mix regions, employer sizes and years of experience so the task list isn't one company's workflow.
2. Define the domains. The panel sets the high-level topic areas. Assessment Systems puts the typical range at 5 to 20 domains. For most professional and product certifications, fewer is easier to live with (more on that in step 3).
3. Write the task list. Each task starts with a verb and describes observable work ("Configure SAML single sign-on for a new tenant," not "Understand SSO"), assigned to a domain.
4. Survey a representative sample of practitioners. Respondents rate each task, usually on importance and time spent (also called criticality and frequency), and answer demographic questions such as region, work setting, years of experience, education and other certifications held.
5. Analyze and report. A psychometrician analyzes the ratings and writes a formal report that the blueprint references.

Accreditation makes this non-optional. Assessment Systems is blunt that programs seeking NCCA or ANSI/ANAB ISO 17024 accreditation are required to do some form of job task analysis.

Survey details that save you a second round

• Define the rating scales in plain words. "Importance: how serious is the harm if this task is done wrong?" and "Frequency: how often do you do this? Never, yearly, monthly, weekly, daily." Vague scales produce ratings you can't interpret.
• Add a "do you perform this task?" screen. A respondent who never does a task shouldn't rate its importance.
• Pilot with a handful of SMEs first. They'll find duplicate tasks, jargon and the point where the survey gets too long to finish.
• Slice results by experience. If entry-level and senior practitioners rate tasks very differently, you may need two credentials.

How often should you repeat the JTA?

The NCCA Standards say a job analysis must be run often enough that the content specifications reflect current practice. The commentary on Standard 14 gives every five years as a general guideline, one to three years for fast-changing fields, and as long as seven to eight years where the work changes slowly. Product and software certifications usually land at the fast end, because the product changes under the exam.

Step 2: how do survey ratings become domain weights?

The mechanics are simpler than they look. For each task, combine the mean importance and mean frequency ratings (Assessment Systems notes programs either add or multiply them). Sum the task scores within each domain, then divide each domain's sum by the grand total. That gives you the suggested weight for each domain.

Here's an illustrative example for a fictional entry-level platform administrator credential. The numbers are made up to show the arithmetic, not taken from any real study.

Domain	Sum of task scores (importance x frequency)	Suggested weight
1. Platform setup and configuration	180	15%
2. User and access management	240	20%
3. Course and content administration	300	25%
4. Assessments and certificates	180	15%
5. Reporting and data	180	15%
6. Troubleshooting and support	120	10%
Total	1,200	100%

The survey gives you a suggestion. The SME panel then reviews it, and there are two adjustments we'd expect them to discuss:

• Rare but high-stakes tasks. Multiplying by frequency pushes down tasks that happen once a year but cause real damage when done wrong (restoring from backup, handling a data request). Panels often protect these with a minimum weight or by giving importance more influence. Document the rule you use so it applies the same way to every domain.
• Tasks you can't test well. Some high-rated tasks can't be measured fairly online. Note them and consider a practical assessment instead.

Keep the adjustments small and written down. If the final weights look nothing like the survey, you've quietly gone back to weights picked in a meeting.

Step 3: how do you set cognitive levels and item counts?

Cognitive level tells you what kind of thinking items demand. A simple two-level split works for most programs: recall (define, identify, recognize) and reasoning (apply, analyze, troubleshoot, choose between options in a scenario).

CEDMA's best practices for certification exam blueprints give a useful starting mix by level: 70% recall and 30% reasoning for entry-level exams, 40% recall and 60% reasoning at mid-level, and 10% recall and 90% reasoning for advanced exams, with the advice to go beyond those reasoning shares where you can. The same article sets practical limits on structure: no more than seven sections and 20 to 22 objectives.

Those limits matter. With a 60-item exam and 40 objectives, most objectives get one or two items, and CEDMA questions whether a single correct answer shows mastery. Fold narrow objectives into broader umbrella objectives rather than stretching the exam and its delivery cost.

To get item counts, multiply total scored items by each domain weight, round, and reconcile the total. Then split each domain's count by cognitive level. Troubleshooting domains usually lean toward reasoning even on an entry exam, and setup domains lean toward recall, so the split doesn't have to be identical in every row. It has to add up at the exam level.

Step 4: how big should the item bank be, and how many forms do you need?

A blueprint for 60 scored items does not mean you write 60 items. You need enough to build more than one form, replace items that perform badly, and limit how often any single item is seen.

PSI's article on how many items you need to create an exam uses item exposure to size the pool. In its example, a 100-item exam taken by 1,500 candidates a year gives each item 750 views with two forms and 375 views with four. If you want no item seen more than 500 times, you need a pool of about 300 items, which PSI frames as three times what the exam specification calls for. PSI also points out that items aren't evergreen. Statistics and content reviews retire them, so technology exams may need new items every six months while slower-moving content might refresh every two years.

Three planning rules we use:

1. Size the bank per domain, not in total. A 3x bank that's heavy on easy domains still leaves you one form deep in the domain that matters most.
2. Write more than the target. SME review rejects items, and field testing rejects more. Plan item writing workshops for a surplus, not the exact count.
3. Set your exposure tolerance before launch. High-volume exams with active prep communities need more forms and faster refresh than a small internal credential.

Sample blueprint table you can copy

Here is the fictional entry-level exam from step 2 carried through to a full blueprint: 60 scored items, six domains, about 20 objectives (roughly three items each), and a 70/30 recall-to-reasoning mix at the exam level, following the CEDMA entry-level guidance linked above. Bank targets use the 3x rule from PSI.

Domain	Weight	Objectives	Scored items	Recall	Reasoning	Bank target (3x)
1. Platform setup and configuration	15%	3	9	7	2	27
2. User and access management	20%	4	12	9	3	36
3. Course and content administration	25%	5	15	11	4	45
4. Assessments and certificates	15%	3	9	7	2	27
5. Reporting and data	15%	3	9	6	3	27
6. Troubleshooting and support	10%	2	6	2	4	18
Total	100%	20	60	42	18	180

Under each domain, list objectives with the job tasks they trace to, such as "2.1 Provision and deprovision users through SSO and SCIM (tasks T14, T15, T19)." That traceability answers "why is this on the exam?" three years from now.

Once SMEs ratify the domains, objectives and item counts, freeze it. CEDMA's wording is that the approved blueprint should be treated as "cast in concrete." Changes wait for the next JTA or a formal, documented review, not for a stakeholder who wants their topic added mid-cycle.

Can you use AI to draft certification exam items?

Yes, with conditions. The NCCA published its guidance document on the use of AI in certification programs (version 1.1, released April 2, 2025), and it's the clearest statement so far of where the lines are for accredited programs:

• AI can help with job analysis, exam specifications and item development, but SMEs must review all AI-generated content for accuracy, fairness and psychometric quality. In job analysis it can help analyze data but must not replace input gathered directly from certificants or the target audience.
• AI can't make essential certification decisions on its own. Pass/fail decisions need audit and validation by qualified people, and passing standards come from professional judgment, not an AI-set threshold.
• Secure content stays away from AI. Programs need policies that stop AI from being trained on or accessing non-public items, scoring rubrics or proprietary exam specifications.
• Disclose it. AI use goes into NCCA initial applications, five-year reaccreditation applications and annual reports, and changes that affect exam development, administration or evaluation need a material change notice.
• Proctoring still needs people. AI can flag irregularities but must not terminate or invalidate an exam by itself.

In practice, the safest workflow drafts from public material: the published blueprint objectives, public product documentation, published standards. An item writer prompts for a scenario-based draft tied to one objective and cognitive level, checks the key and distractors against the source, and sends it through normal SME review. Treat the draft like a first pass from a junior writer.

Watch the second-order risk. Tools that audit an existing bank send those items to the model provider. If your live bank is secure content under NCCA's definition, run that kind of tool only on retired or draft items, or confirm with your psychometrician and accreditor first. Self-hosting the LMS doesn't change this if the AI calls still go out to a third-party model.

In LMS Advisor, the AI features run on Anthropic or OpenAI (the admin picks), can be switched off entirely, and never auto-publish a draft. AI question generation, question-bank audit and auto-tagging are separate tools, so a program can use generation on public material and leave the audit off for its secure bank.

How do you load the blueprint into your exam platform?

The blueprint lives in a document and the bank lives in your exam platform. Drift starts in the gap. A setup that holds up:

1. Tag every item with its domain, objective and cognitive level when it's written. Retro-tagging a few hundred items is a week nobody wants.
2. Organize pools by domain so form assembly and bank-depth checks happen domain by domain.
3. Use difficulty levels so forms stay comparable when the items differ.
4. Keep scored items in objectively scored formats. Check how your platform treats essays and uploads. In LMS Advisor, essay and file-upload exam questions are screening items and don't count toward the score, so a blueprint row can't be met with them.
5. Check every form against the blueprint before release. Count items per domain and per cognitive level and compare with the table. Put this in your release checklist.

Be honest about what your platform does automatically. Ask the vendor which of these steps the software handles and which stay manual. Step 5 is the one most often left to a person.

How to run your exam blueprint in LMS Advisor

We built the Test Center for programs that start from a blueprint like the one above, so each part of the table has a place to live:

Blueprint element	Where it goes in LMS Advisor
Domains	One pool per domain in the question bank, so bank depth is visible domain by domain
Objectives and cognitive levels	Item tags, with AI auto-tagging to speed up tagging an existing bank (review the tags before you rely on them)
Comparable forms	Difficulty levels on every item, plus negative marking if your scoring model uses it
Reasoning items	14 question types, including coding, image hotspot, audio answers and video answers
Item drafting	AI question generation for drafts from public material, never auto-published, and switched off for secure content
Delivery	A standalone Test Center exam with its own candidate portal, email invitations or one-time-code access, and no course enrollment needed
Exam security	Native proctoring set per exam, with an evidence trail for reviews and disputes
Credential	A certificate issued automatically on pass, with a public verification page an employer can check

Two limits to plan around. LMS Advisor doesn't enforce per-domain blueprint counts for you, and it doesn't do linear-on-the-fly (LOFT) or computer adaptive testing, so the form check in step 5 stays a human task. If LOFT or adaptive delivery is part of your design, you need a dedicated testing platform for that piece. Exam certificates can carry a validity period, and if your credential renews by retraining, pair the exam with a recertification course whose certificate renews on a retake.

For a wider view of how platforms handle standalone candidates, proctoring and renewals, see our comparison of the best LMS for certification programs, and if you're building a credential for resellers, our guide to running a partner certification program covers tiers and renewals. Our exam and certification program setup and the page for certification bodies show the full workflow, or start a trial and load your first domain's pool.

Frequently asked questions

Can LMS Advisor deliver a blueprint-based certification exam?

Yes. You build one question pool per domain, tag items by objective and difficulty, and deliver the exam through the Test Center's candidate portal with native proctoring and a verifiable certificate on pass. Checking each form against the blueprint counts stays a manual step, and LOFT or adaptive testing needs a dedicated testing platform.

What is the difference between a job task analysis and an exam blueprint?

A job task analysis describes the job, and the blueprint describes the exam. The JTA produces practitioner ratings for each task, and the blueprint turns them into domain weights, objectives, item counts and cognitive levels.

How many questions per domain should a certification exam have?

Multiply the total number of scored items by each domain's weight from the JTA, then round and reconcile the total. A domain weighted at 20% on a 60-item exam gets 12 items. Also aim for at least two or three items per objective, because CEDMA's blueprint guidance questions whether a single item shows mastery.

How large should a certification item bank be?

A common target is about three times the number of scored items on one form, sized domain by domain. PSI uses that 3x figure in its exposure example for a 100-item exam with 1,500 candidates a year. High-volume or fast-changing exams need more.

Do you need a job task analysis for NCCA or ISO 17024 accreditation?

Yes. NCCA Standard 15 requires the exam's weighting plan to be based on a job analysis, and Assessment Systems notes that ANSI/ANAB ISO 17024 accreditation requires some form of job task analysis too. The NCCA commentary suggests repeating it about every five years, more often in fast-changing fields.

Can certification bodies use AI to write exam questions?

Yes, as a drafting aid with human validation. NCCA's April 2025 guidance allows AI to assist item development but requires SMEs to review all AI-generated content, bars AI from training on or accessing secure test materials, and requires programs to disclose AI use to NCCA.

Should the exam blueprint be published?

Publish the content outline: domains, weights and objectives, plus exam length and format. NCCA Standard 6 expects programs to make the content domains and weightings public. Keep item counts per objective, bank size and form assembly rules internal.