Teaching the dissertation with one case, three ways

Session	Steps	Teaching time
Session 1: Frame the study	1.1 to 1.6	About 47 min, plus discussion
Session 2: Build the argument	1.7 to 2.2	About 55 min, plus discussion
Session 3: Design the method	3.1 to 3.6	About 56 min, plus discussion
Session 4: Report and act	4.1 to 6	About 38 min, plus discussion

Jump to a step

Chapter 1: Introduction
1.1Background
1.2Problem statement
1.3Sub-problems
1.4Purpose
1.5Research objectives (SMART)
1.6Research questions
1.7Design objectives and design propositions
1.8Conceptual framework
1.9Scope, assumptions and limitations
1.10Contribution to praxis
Chapter 2: Review of literature
2.1Literature themes mapped to research questions
2.2Research and practitioner gap
Chapter 3: Research design and methods
3.1Research design
3.2Justification of the approach
3.3Data and sample
3.4Instruments and ethics
3.5Data collection plan
3.6Statistical methods and evaluation metrics
Chapter 4: Results and findings
4.1Results and findings (illustrative)
Chapter 5: Conclusions, recommendations and further research
5.1Conclusions
5.2SMART recommendations
5.3Areas for further research
Before you submit
6Traceability and submission check

Chapter 1: Introduction · Step 1 of 23 · about 8 min

1.1Background

Teach this

The opening must let a stranger name the organisation, the technology and the worry in three sentences. Everything else earns its place only if it supports the problem.

Worked example: PSU

PSU has deployed AI in five functions: admissions screening, a learning-analytics dashboard, a vendor-run student chatbot, an AI-writing detector and a generative-AI research assistant. Each handles personal or research data, and three are run by outside vendors. (Cite 2–3 sources on sector adoption here.)

These systems add attack surfaces that conventional IT security only partly covers: prompt injection, leakage of training or prompt data, model inversion and dependence on vendors (cite). India's Digital Personal Data Protection Act 2023 raises PSU's duties as a data fiduciary, and international students add GDPR exposure (cite). Generic guidance exists (NIST AI RMF, ISO/IEC 27001), but each university has to translate it into its own setting.

PSU's leadership reports concern but has no shared way to assess, compare or govern AI security across systems. That is a need for a designed, evaluated solution, not another descriptive survey. PSU here stands for a public university, used as an illustration.

Do

Move from the global trend to PSU using dated, credible evidence: regulator reports, peer-reviewed studies and PSU documents.
Frame it as a business problem with scholarly relevance. A DBA needs both.
Confirm access to PSU data, people and documents before you commit to the topic.

Don't

Write a general essay on AI in the world.
Pick a topic whose data PSU may not release.
Use vendor blogs or press releases as core evidence.

Mistakes students make here

Student writes: "AI is transforming every industry, and security matters more than ever."
Why it fails: Generic. The sentence could open any paper on any organisation.
Fix: Name PSU's five systems, the kind of data they hold and the rule that raises liability (DPDP Act 2023).

Student writes: Two pages on the history of AI.
Why it fails: Uses the word budget on background that does not lead to the problem.
Fix: Cap history at two sentences. Keep it only if it answers a research question.

Student writes: Cites a vendor blog and a news article as the main evidence.
Why it fails: Weak evidence for an academic audience, and easy for an examiner to challenge.
Fix: Use regulator reports, peer-reviewed studies and one PSU document that proves the adoption.

SMART check for this step

S: SpecificNames PSU, the AI systems in use and the security concern.
M: MeasurableAt least 10 credible sources, half from the last five years, plus one PSU document showing the adoption.
A: AchievableWritten access confirmation from a PSU gatekeeper.
R: RealisticFits the 8–10 page topic proposal.
T: Time-boundDrafted before Immersion 1.

In-class check

Give students 5 minutes to rewrite their opening paragraph so that a stranger can name the organisation, the technology and the worry.

Pair check: the partner underlines every sentence that could appear in any other paper. Delete or replace them.

Chapter 1: Introduction · Step 2 of 23 · about 10 min

1.2Problem statement

Teach this

A problem statement says what is happening, what should be happening, who is affected and what is unknown. It never names the fix. In a DSR study the artefact is the answer, so it cannot already be in the question.

Worked example: PSU

PSU lacks a validated, usable method to assess and govern security across its AI systems.

Practitioner gap: the CISO and unit heads cannot compare systems, assign accountability or prioritise investment, so decisions rest on anecdote. Research gap: existing frameworks are generic and rarely evaluated with users in universities, so evidence that any artefact works in this setting is thin.

Do

Write one paragraph: what is happening, what should be happening, who is affected, what is unknown.
Split the gap into a practitioner gap and a research gap.
Keep the problem solvable with data you can reach.

Don't

Name a technology fix inside the problem ("PSU needs an AI firewall").
Write moral or vague statements ("AI is dangerous").
Choose a problem too large for the time you have.

Mistakes students make here

Student writes: "PSU needs an AI security framework."
Why it fails: The solution is inside the problem. There is nothing left to design or test.
Fix: State the missing capability: PSU cannot assess, compare or govern security across its AI systems.

Student writes: "The problem is AI security in universities."
Why it fails: That is a topic, not a problem. Nobody can tell what is wrong, for whom or what is unknown.
Fix: Use the four-part test: what is, what should be, who is affected, what is unknown.

Student writes: Only a practitioner gap, or only a research gap.
Why it fails: A DBA dissertation has to matter to practice and to scholarship.
Fix: Write one sentence for each and link them.

SMART check for this step

S: SpecificOne population, one setting, one paragraph.
M: MeasurableThe gap is evidenced by at least 3 sources and one PSU fact.
A: AchievableThe data needed to address it is accessible.
R: RealisticSolvable at doctoral scale.
T: Time-boundSigned off by your chair before Milestone 1.

In-class check

Delete test: read the problem statement aloud without its last sentence. Can the class still say what needs to be built? If not, the statement is too vague.

Chapter 1: Introduction · Step 3 of 23 · about 6 min

1.3Sub-problems

Teach this

Sub-problems break the problem into parts that can each be answered. Each one becomes exactly one objective and one question later, so orphans and overlaps are defects.

Worked example: PSU

SP1 The requirements for AI security governance at PSU are undocumented.
SP2 No artefact exists that meets those requirements across the AI lifecycle.
SP3 There is no evidence that a candidate artefact is accurate, usable and effective in PSU's setting.

Do

Make each sub-problem a component of the main problem.
Keep them to 3–5, each mapped to one objective.
Phrase each as a gap, not as a solution.

Don't

List overlapping sub-problems that need the same data.
Add issues you will not research (cost, bias, ethics).
Phrase a sub-problem as a solution.

Mistakes students make here

Student writes: "SP1 Introduction. SP2 Literature. SP3 Methods."
Why it fails: These are chapter headings, not parts of the problem.
Fix: Write each as a gap that PSU has: undocumented requirements, no artefact, no evidence.

Student writes: "SP1 Develop a maturity model."
Why it fails: That is a task, or an objective, not a gap.
Fix: Rephrase as the gap: no artefact exists that meets the requirements.

Student writes: Five sub-problems for three objectives, two of them overlapping.
Why it fails: The mapping SP → objective → question breaks, so the design is not traceable.
Fix: Merge overlaps and keep 3–5.

SMART check for this step

S: SpecificEach states one issue.
M: MeasurableEach traces to exactly one research objective.
A: AchievableEach has an accessible data source.
R: Realistic3–5 in total.
T: Time-boundFixed before you write the objectives.

In-class check

Draw three columns on the board: SP, RO, RQ. Ask students to fill them from their own drafts. Any empty cell or duplicate is a defect to fix before the next step.

Chapter 1: Introduction · Step 4 of 23 · about 5 min

1.4Purpose

Teach this

One sentence: a verb, the design, the population and the setting. The verb has to match the design. A DSR study designs, demonstrates and evaluates. It does not "explore".

Worked example: PSU

The purpose of this design science study is to design, demonstrate and evaluate an AI security governance artefact, comprising a maturity model and a control toolkit, for PSU.

The study produces two things: a validated artefact for PSU and a set of design principles other universities can reuse.

Do

Use one sentence with a verb, the design, the population and the setting.
Match the wording to your design (explanatory, quantitative, design science).
State what the study will produce.

Don't

Write "to explore" when you will test hypotheses.
Promise outcomes the design cannot deliver ("to eliminate risk").
Leave the setting or population vague.

Mistakes students make here

Student writes: "The purpose is to explore AI security at PSU."
Why it fails: "Explore" belongs to exploratory qualitative work. It hides the artefact and the evaluation.
Fix: Use design, demonstrate and evaluate.

Student writes: A five-line purpose with three aims in it.
Why it fails: A purpose that cannot be said in one breath will not guide the design.
Fix: One sentence. Move the rest to the objectives.

Student writes: "...to improve security at PSU."
Why it fails: Promises an outcome the study cannot measure in 8 months.
Fix: State what the study produces: a validated artefact and design principles.

SMART check for this step

S: SpecificVerb, design, population and setting all named.
M: MeasurableThe output can be checked: a model, a test result or a framework.
A: AchievableConsistent with the data you can reach.
R: RealisticAchievable within the programme timeline.
T: Time-boundApproved by your chair at Milestone 1.

In-class check

One-breath test: each student reads their purpose statement aloud in one breath. If they run out of air, it is too long.

Chapter 1: Introduction · Step 5 of 23 · about 10 min

1.5Research objectives (SMART)

Teach this

An objective is a promise you can be graded on: a measurable verb, a number and a deadline counted from t0. Use "t0 + months" so an IRB delay does not silently break the plan.

Worked example: PSU

Objective (SP, RQ)	Specific	Measurable	Achievable	Realistic	Time-bound
RO1 Elicit requirements (SP1, RQ1)	Requirements for AI security governance from PSU stakeholders	12 or more interviews; requirement list agreed by at least 80% of participants	Interviewees identified through PSU committees	45-minute interviews	t0 + 2 months
RO2 Design and build (SP2, RQ2)	Maturity model and toolkit across four AI lifecycle phases	At least 25 controls, each mapped to a NIST AI RMF or ISO 27001 clause	Builds on published frameworks	Existing frameworks act as the kernel; two build cycles only	t0 + 5 months
RO3 Demonstrate and evaluate (SP3, RQ3)	Apply to PSU AI systems; expert and user evaluation	3 systems; 8 or more experts with I-CVI of 0.78 or above; 15 or more users with mean SUS of 68 or above	Experts reached through professional networks	Two evaluation rounds, not more	t0 + 8 months

Do

Start each objective with a measurable verb: identify, measure, test, design, evaluate.
Count deadlines from a fixed event such as IRB clearance (t0), not from a calendar date you cannot control.
Keep objective, research question and sub-problem one-to-one.

Don't

Use "understand", "explore" or "look into" for objectives you will test.
Promise outcomes the design cannot deliver.
Add an objective that has no data behind it.

Mistakes students make here

Student writes: "To understand the impact of AI on security at PSU."
Why it fails: "Understand" cannot be measured, so nobody can tell whether it was achieved.
Fix: Use a verb that leaves evidence: elicit, design, evaluate. Add a number.

Student writes: "...by December 2027."
Why it fails: A fixed calendar date breaks when IRB approval slips.
Fix: Count from t0: "t0 + 5 months".

Student writes: "To reduce security incidents at PSU."
Why it fails: The study cannot show incident reduction in 8 months. The objective promises what the data cannot deliver.
Fix: Aim at what the study can measure now: accuracy, usability and risks found.

SMART check for this step

S: SpecificOne verb, one outcome, one population.
M: MeasurableA number, a threshold or a named test.
A: AchievableData and access already confirmed.
R: RealisticFits your resources and the programme timeline.
T: Time-boundDeadline counted from t0.

In-class check

Have students underline the verb, the number and the deadline in each objective. Missing any of the three means it is not SMART yet.

Ask: which column of the SMART table is hardest to fill in for your own objective? That column shows what needs more thought.

Chapter 1: Introduction · Step 6 of 23 · about 8 min

1.6Research questions

Teach this

Questions are what the study answers. One per objective, open in form, and answerable with data the student can name today.

Worked example: PSU

Research question	Objective	Sub-problem	Answered by
RQ1 What requirements should AI security governance meet at PSU?	RO1	SP1	Interviews and document review
RQ2 How can a maturity model and control toolkit be designed to meet those requirements?	RO2	SP2	Design cycles, kernel theories
RQ3 How accurate, usable, effective and useful is the artefact in PSU settings?	RO3	SP3	Expert panel, user evaluation, risk logs

Do

Ask open questions: what, how, to what extent.
Keep them to 3–5, one per objective.
Put question, objective and hypothesis in one traceability table.

Don't

Ask yes/no questions ("Is AI secure?").
Ask what your data cannot answer.
Mix two constructs in one question.

Mistakes students make here

Student writes: "Is AI secure at PSU?"
Why it fails: Yes/no question. It does not lead to a design or a measure.
Fix: Ask what, how or to what extent: what requirements should governance meet?

Student writes: Five questions for three objectives.
Why it fails: Two questions have no objective behind them and no data to answer them.
Fix: Keep one per objective.

Student writes: "What is the best framework for AI security?"
Why it fails: "Best" is undefined, so no evidence can settle it.
Fix: Ask how a framework can meet stated requirements and how well it does.

SMART check for this step

S: SpecificOne construct or artefact per question.
M: MeasurableYou can name the data and test that answers it.
A: AchievableAnswerable with data from PSU.
R: RealisticAnswerable within your timeline.
T: Time-boundTied to the objective's deadline.

In-class check

Swap and name the data: students exchange their research questions. The neighbour writes, beside each one, the data that would answer it. A blank means the question cannot be answered.

Chapter 1: Introduction · Step 7 of 23 · about 12 min

1.7Design objectives and design propositions

Teach this

In DSR you do not hypothesise about people. You state what the artefact must do (design objectives), then how you will know it did (design propositions with a metric and threshold). Students trained on surveys often need this reset.

Worked example: PSU

A design study does not test hypotheses about people. It states design objectives from the requirements and design propositions that test whether the artefact meets them.

Design objectives (from RO1)

DO1 Every AI system has a named accountable owner.
DO2 The artefact covers the full AI lifecycle: data, model, deployment and operation.
DO3 Every control maps to a NIST AI RMF, ISO/IEC 27001 or DPDP clause.
DO4 A unit head can complete an assessment in 2 hours or less.
DO5 Scores are comparable across systems.

Design propositions (tested in RO3)

Proposition	Claim	Metric and decision rule
DP1 Accuracy	Experts judge the maturity levels and controls accurate and relevant	I-CVI of 0.78 or above per control; S-CVI/Ave of 0.90 or above; Kendall's W of 0.6 or above
DP2 Usability	Unit heads rate the toolkit usable	Mean SUS of 68 or above
DP3 Effectiveness	The toolkit finds more risks than the last IT review of the same system	At least 30% more distinct, validated risks per system on average, adjudicated by the CISO
DP4 Utility	Users find it useful for prioritising investment	At least 80% rate 4 or 5 on a 5-point item

Do

Derive design objectives from the requirements, then write design propositions that test the artefact against them.
Give every proposition a metric and a decision rule (for example, I-CVI of 0.78 or above).
Register the propositions in the proposal before evaluation starts.

Don't

Claim the artefact "solves" AI security. Claim that it meets stated objectives.
Write propositions you cannot test with the evaluators and systems you have.
Change decision rules after you see the evaluation results.

Mistakes students make here

Student writes: "H1: The artefact will improve AI security at PSU."
Why it fails: No metric, no threshold, no way to fail. Not testable.
Fix: "DP1: experts rate the controls accurate: I-CVI of 0.78 or above."

Student writes: Thresholds chosen or changed after the evaluation.
Why it fails: Moving the goalposts. It invalidates the evaluation and is a serious integrity issue.
Fix: Fix thresholds in the proposal and cite a benchmark (CVI, SUS).

Student writes: "H1: Perceived usefulness positively influences adoption."
Why it fails: That is a survey hypothesis about people, copied from a different design.
Fix: Write propositions about the artefact's properties: accuracy, usability, effectiveness, utility.

SMART check for this step

S: SpecificOne artefact property per proposition: accuracy, usability, effectiveness or utility.
M: MeasurableA metric and threshold for each.
A: AchievableEvaluators and systems already identified.
R: RealisticThresholds based on published benchmarks (CVI, SUS).
T: Time-boundFixed before Cycle 1 begins.

In-class check

Failure question: for each proposition, ask what result would make you say the artefact failed. If the student cannot answer, the proposition is not falsifiable.

Chapter 1: Introduction · Step 8 of 23 · about 12 min

1.8Conceptual framework

Teach this

The framework is the audit trail from theory to requirement to design principle to feature. If a feature cannot be traced back, it is decoration. Show the build-and-evaluate loop in the figure.

Worked example: PSU

The framework is a chain from kernel theory to requirement to design principle to feature. Each row is one link you must be able to defend.

Requirement (from RO1)	Kernel theory	Design principle	Feature in the artefact
REQ1 Accountability must be assignable	NIST AI RMF: Govern	DPr1 Every AI system has a named owner	Ownership field and RACI template per system
REQ2 Cover the whole AI lifecycle	NIST AI RMF: Map and Measure	DPr2 Assess data, model, deployment and operation	Four lifecycle phases, each with its own controls
REQ3 Vendors must be transparent	Trust theory; ISO 27001 supplier controls	DPr3 Make vendor duties explicit and checkable	Vendor checklist: breach notice, data location, audit rights
REQ4 Reuse existing audits	ISO/IEC 27001	DPr4 Map every control to a recognised standard	Clause reference on each control
REQ5 Light enough for unit heads	Usability principles	DPr5 Keep assessment to 2 hours or less	One scoring sheet, 25–40 controls, guided by yes/partly/no
REQ6 Comparable across systems	Maturity model logic	DPr6 Score all systems on the same four levels	Level 1–4 scoring rule and summary radar

Do

Show the chain from kernel theory to requirement to design principle to feature.
Name the kernel theories and say which part of the artefact each one informs.
Show the build-and-evaluate loop in your framework figure.

Don't

Copy NIST AI RMF or ISO 27001 controls wholesale without adapting them to PSU.
Add a feature with no requirement behind it.
Leave design principles as slogans with no link to evidence.

Mistakes students make here

Student writes: A box diagram of the artefact's components, with no theory.
Why it fails: It shows what was built, not why. Examiners cannot see the rigour.
Fix: Add kernel theories and trace each feature to a requirement.

Student writes: Pasting NIST AI RMF or ISO 27001 controls unchanged.
Why it fails: Copying a standard is not design, and it does not fit PSU's context.
Fix: Adapt each control to PSU, map it to a clause and record the reason for changes.

Student writes: A feature with no requirement behind it.
Why it fails: Unjustified features expand scope and cannot be evaluated.
Fix: Every feature traces to one requirement. Remove the rest.

SMART check for this step

S: SpecificEvery feature traces to one requirement and one kernel theory.
M: MeasurableAt least 25 controls, each mapped to one standard clause.
A: AchievableBuildable with PSU's resources.
R: RealisticScaled to two build cycles.
T: Time-boundFrozen before Cycle 1.

In-class check

60-second trace: pick one feature from the toolkit. On the board, trace it back to a requirement, a design principle and a kernel theory. If the chain breaks, discuss whether to add the missing link or drop the feature.

Chapter 1: Introduction · Step 9 of 23 · about 6 min

1.9Scope, assumptions and limitations

Teach this

Scope is a promise about what the student will not do. Limitations are honest constraints, each with a mitigation. Separate the two: scope is a choice, a limitation is a constraint.

Worked example: PSU

In scope

AI systems in production at PSU for at least 6 months (five systems; three used for demonstration).
Four lifecycle phases: data, model, deployment and operation.
Security only: confidentiality, integrity, availability and privacy.

Out of scope

Penetration testing and vendor-side model internals.
AI ethics and bias, except where they create a security risk.

Assumptions

PSU grants access to systems, policies and staff.
Experts give candid ratings.
Unit heads score their own systems honestly.

Limitations and mitigations

Limitation	Mitigation
Single institution	Generalise through design principles, not statistics (Gregor and Hevner, 2013)
Self-assessment can be gamed	Spot-check with a document review and CISO adjudication
Evaluation lasts 8 months, so incident reduction cannot be shown	Measure proxy outcomes (risks found, usability, utility) and name incident tracking as further research

Do

Say what is in, what is out and why.
Give each limitation a mitigation.
Separate delimitations (your choices) from limitations (constraints).

Don't

Hide weaknesses. Examiners find them.
Apologise for a limitation. State it and mitigate it.
Let the scope grow during data collection.

Mistakes students make here

Student writes: "The study covers AI security at PSU."
Why it fails: No boundary. Any finding or gap can be added later, so the work never ends.
Fix: Name the systems, phases and period, and say what is out (penetration testing).

Student writes: "Limitations: time and resources."
Why it fails: Generic, and gives the reader nothing to judge.
Fix: Name the specific limitation (one institution, self-assessment) and how it is mitigated.

Student writes: Listing delimitations as limitations, or the reverse.
Why it fails: Blurs what the student chose from what was forced on them.
Fix: Delimitation: choice (no penetration testing). Limitation: constraint (single site).

SMART check for this step

S: SpecificEach boundary is explicit: systems, groups, period.
M: MeasurableEach limitation lists a named mitigation.
A: AchievableThe scope matches your access.
R: RealisticThe scope fits the time and the word limit.
T: Time-boundLocked at Milestone 2 and changed only with your chair's approval.

In-class check

Each student states one thing their study will not do, and the reason. The class checks that it is a boundary, not an excuse.

Chapter 1: Introduction · Step 10 of 23 · about 5 min

1.10Contribution to praxis

Teach this

A DSR dissertation makes two contributions: knowledge (design principles that others can reuse) and practice (the artefact PSU can use). Naming only the artefact is the most common gap.

Worked example: PSU

Theory: prescriptive design knowledge in the form of six design principles and a tested mapping from NIST AI RMF and ISO 27001 into a university setting. This closes the research gap on evaluated, university-specific AI security artefacts.

Practice: a toolkit PSU can adopt at once, and a scoring method that lets leaders compare systems and target investment. Other universities can adapt it.

Do

State the theoretical contribution and the practical contribution separately.
Tie each contribution to the gap you named.
Say who will use the result at PSU and beyond.

Don't

Overclaim generalisation from one institution.
List contributions your results cannot support.
Confuse a recommendation with a contribution.

Mistakes students make here

Student writes: "This framework can be used by all universities."
Why it fails: Overclaims. One institution cannot support that.
Fix: Say principles may transfer, and name how you would test it.

Student writes: Recommendations listed as contributions.
Why it fails: Recommendations are for PSU. Contributions are what the field learns.
Fix: Separate them: contributions in Chapter 1, recommendations in Chapter 5.

Student writes: Only the artefact is named; there is no design knowledge.
Why it fails: Without design principles, DSR is just a consulting project.
Fix: State the principles and link to Gregor and Hevner (2013).

SMART check for this step

S: SpecificOne theoretical and one practical contribution, each named.
M: MeasurableEach is tied to a specific finding or artefact you can point to.
A: AchievableWithin what your design can support.
R: RealisticScaled to a single-institution study.
T: Time-boundConfirmed in Chapter 5 after the results.

In-class check

Ask: what would a reviewer learn from this study if they never met PSU? The answer is the scholarly contribution.

Chapter 2: Review of literature · Step 11 of 23 · about 15 min

2.1Literature themes mapped to research questions

Teach this

The review is an argument organised by theme, not a list of authors. Every theme has people who support the student's propositions, people who challenge them and people who are neutral. It ends by pointing at the gap.

Worked example: PSU

Theme (RQ)	Supports your propositions	Challenges	Neutral	Critical evaluation
AI threat landscape in education (RQ1)	AI creates new attack vectors that need dedicated requirements	Most threats are familiar cyber risks under new names	Severity depends on architecture and vendor	Mostly conceptual or vendor-sponsored; few university cases
Governance frameworks and maturity models (RQ2)	Structured frameworks (NIST AI RMF, ISO 27001) improve accountability	Maturity models become checklists and fit universities poorly	Value depends on tailoring	Rarely validated with users
Design science and evaluation (RQ2, RQ3)	Building artefacts yields transferable design knowledge (Hevner et al., 2004; Peffers et al., 2007)	DSR is criticised as consulting with weak theory	Rigour depends on the evaluation (Venable et al., 2016)	Many artefacts are only demonstrated, not evaluated
Usability and adoption of security tools (RQ3)	Usability drives adoption	Mandates can drive use regardless of usability	Depends on context	SUS is generic and says little about security value
Regulation and privacy: DPDP, GDPR (RQ1)	Law forces protection	Compliance is not security	Impact varies by jurisdiction	Legal analysis rarely tests operational effect

Add at least two named sources in each cell from your database search. Anchor works: Hevner et al. (2004), Peffers et al. (2007), Venable et al. (2016), NIST (2023).

Do

Organise by theme and map each theme to one research question.
For each theme show authors who support, challenge and are neutral to your propositions.
Evaluate the studies (sample, method, context, bias) and end with the gap.

Don't

Write an annotated bibliography ("Smith said… Jones said…").
Cite only papers that agree with you.
Cite from memory or from AI output without opening the paper. AI tools can invent references.

Mistakes students make here

Student writes: "Smith (2022) studied AI. Jones (2021) studied security. Lee (2020) studied universities."
Why it fails: Annotated bibliography. There is no argument, comparison or link to a question.
Fix: Group by theme and compare studies: what they agree on, where they conflict.

Student writes: Only sources that agree with the student.
Why it fails: The guide requires supporters, challengers and neutrals. Missing challengers signals weak searching.
Fix: Search for the strongest opposing view in each theme and address it.

Student writes: Cites papers that were never opened, including AI-generated references.
Why it fails: AI tools can invent authors, titles and page numbers. Invented citations are an integrity breach.
Fix: Open every source. Keep a search log with the database and date.

SMART check for this step

S: SpecificEach theme answers one research question.
M: MeasurableAt least 2 sources per position (support, challenge, neutral) per theme; half from the last 5 years.
A: AchievableSources available through your library.
R: Realistic12–18 pages for the full review.
T: Time-boundSearch log and theme matrix complete before Milestone 1.

In-class check

Twenty-minute exercise: assign each student one theme. They find one supporting, one challenging and one neutral source from the library and write one line on how each relates to their proposition.

Spot-check three references live: can the class find the paper?

Chapter 2: Review of literature · Step 12 of 23 · about 5 min

2.2Research and practitioner gap

Teach this

The gap is what the literature cannot yet tell PSU. A reliable template is: "The literature tells us X, but not Y." It must connect to the problem statement and the research questions.

Worked example: PSU

The literature offers generic AI security frameworks and general design science guidance, but it does not offer an evaluated, university-specific artefact that assigns accountability across the AI lifecycle and is light enough for unit heads to use. PSU therefore has no evidence to choose or trust any tool. This study designs and evaluates one.

Do

State the gap as what the literature cannot yet tell PSU.
Link it back to the problem statement.
Keep it to one paragraph.

Don't

Claim nobody has ever studied the topic. Say what is missing in your context.
Invent a gap by ignoring existing work.
Leave the gap disconnected from your research questions.

Mistakes students make here

Student writes: "No one has ever studied AI security in universities."
Why it fails: Absolute claims are almost always false and easy to refute.
Fix: Say what is missing in the specific context: evaluated, university-specific artefacts.

Student writes: A gap that has nothing to do with the research questions.
Why it fails: The study then answers a different question from the one the literature raised.
Fix: Check that each RQ closes part of the gap.

Student writes: Presenting novelty as the gap: "AI is new, so there is little research."
Why it fails: Newness of a topic is not a research gap.
Fix: Name the missing evidence: what has not been tested, with whom, and how.

SMART check for this step

S: SpecificOne gap, named precisely.
M: MeasurableSupported by at least 3 rows of the theme matrix.
A: AchievableAnswerable by your study.
R: RealisticSized to your design.
T: Time-boundWritten before Milestone 1.

In-class check

Each student completes the sentence: "The literature tells PSU ____ but not ____." Read three aloud. The class judges whether the second blank is specific enough to be answered by the study.

Chapter 3: Research design and methods · Step 13 of 23 · about 12 min

3.1Research design

Teach this

Name the methodology, list its steps and show the output of each. In DSR, evaluation is planned before building begins, and demonstration is not the same as evaluation.

Worked example: PSU

DSRM step	What you do at PSU	Output	Deadline
1 Identify the problem	Document PSU's problem; review the five AI systems and the literature	Problem statement, SP1–SP3	t0
2 Define objectives	12 requirements interviews and a document review	Requirements REQ1–REQ6, design objectives DO1–DO5	t0 + 2 months
3 Design and develop	Build maturity model (4 phases, 4 levels) and toolkit (25 or more controls, scoring sheet, RACI template)	Prototype v1, then v2	t0 + 5 months
4 Demonstrate	Apply to three AI systems: chatbot, analytics dashboard, research assistant	Completed assessments	t0 + 7 months
5 Evaluate	Cycle 1: expert review (artificial, ex ante). Cycle 2: user evaluation and risk comparison (naturalistic, ex post)	CVI, SUS, uplift and utility data	t0 + 8 months
6 Communicate	Dissertation, design principles, PSU briefing	Chapters 4–5	t0 + 9 months

Evaluation follows FEDS: a cheap, early artificial evaluation to find design flaws, then a naturalistic evaluation with real users and real systems to test effect. Log every design change with its reason.

Do

Follow a named methodology (Peffers et al., 2007) and show each step's output.
Plan evaluation with FEDS: what is evaluated, when and by whom.
Plan two build-and-evaluate cycles and log every design change with its reason.

Don't

Present demonstration as evaluation. A demonstration shows it works once. An evaluation tests it against criteria.
Skip the rigour cycle: tie design choices to the knowledge base.
Build first and look for a problem afterwards.

Mistakes students make here

Student writes: "This is design science research" followed by a survey plan.
Why it fails: The label does not make it DSR. There is no artefact, no iteration and no evaluation.
Fix: Show the six DSRM steps and the output of each.

Student writes: Presenting one demonstration as the evaluation.
Why it fails: A demonstration shows the artefact can work once. An evaluation tests it against criteria.
Fix: Use FEDS: an early artificial evaluation, then a naturalistic one with real users.

Student writes: Building first and deciding the criteria later.
Why it fails: Criteria chosen after the build tend to fit the result.
Fix: Define propositions and thresholds before Cycle 1.

SMART check for this step

S: SpecificSix DSRM steps, each with a named output.
M: MeasurableEvery design proposition maps to one evaluation activity.
A: AchievableEach activity is feasible with the stated participants.
R: RealisticTwo cycles fit the 8-month build window.
T: Time-boundEach step is dated from t0.

In-class check

Ask students to fill in a two-column table: each design proposition and the evaluation activity that tests it. A proposition with no activity, or an activity with no proposition, shows a gap in the design.

Chapter 3: Research design and methods · Step 14 of 23 · about 8 min

3.2Justification of the approach

Teach this

Defend the method against the obvious alternatives, one sentence each. Examiners ask why not a survey, a case study or action research. A DBA candidate needs a ready answer.

Worked example: PSU

DSR fits because the questions ask how to build something for PSU and whether it works. The philosophy is pragmatism with a design orientation: knowledge is judged by whether the artefact works in use.

Alternative considered	Why it was rejected
Survey of stakeholders	Measures perceptions and does not produce a tool PSU can use
Case study	Describes governance but does not build or test an artefact
Action research	Embeds the researcher in PSU's change process, which costs independence and time
Quantitative model test (for example, PLS-SEM)	Tests relationships between constructs, not whether a designed tool is accurate, usable and effective

Do

Explain why DSR fits: the questions ask how to build something and whether it works.
Say why a survey, case study and action research were rejected.
State your philosophy (pragmatism, design orientation) and what it means for the evidence you accept.

Don't

Choose DSR because it avoids statistics.
Treat the artefact as the only contribution. Design knowledge is the scholarly contribution.
Skip the limits: single site, short evaluation.

Mistakes students make here

Student writes: "DSR was chosen because it is practical."
Why it fails: No comparison. Any method can be called practical.
Fix: Say the questions ask how to build something and whether it works, and state why alternatives cannot answer that.

Student writes: Choosing DSR to avoid statistics.
Why it fails: It is a weak reason and untrue: DSR needs rigorous evaluation.
Fix: Choose it because the RQs call for an artefact and its evaluation.

Student writes: Copying a philosophy paragraph from a textbook.
Why it fails: Turnitin flags it, and the student cannot explain it in the viva.
Fix: Write two sentences on pragmatism in the student's own words and say what it means for the evidence.

SMART check for this step

S: SpecificThree or more alternatives rejected with one reason each.
M: MeasurableEach reason cites a methods source.
A: AchievableThe methods skills exist or are trained.
R: RealisticFits one page.
T: Time-boundWritten before Milestone 1.

In-class check

Role play: pair students. One defends DSR, the other plays a survey researcher and asks "why not just survey people?" Swap after three minutes.

Chapter 3: Research design and methods · Step 15 of 23 · about 8 min

3.3Data and sample

Teach this

Every number needs a reason and every participant needs a criterion. Evaluators should be able to say the artefact is wrong. A friendly panel gives a comfortable but worthless evaluation.

Worked example: PSU

Group	n	Sampling and criteria	Why this size
Requirements interviews	12	Purposive, maximum variation: CIO, CISO, data protection officer, 3 faculty, 2 students, 2 administrators, 1 vendor manager	Covers all roles; stop early if no new requirements appear
Documents	All relevant	NIST AI RMF, ISO 27001, DPDP Act, PSU policies, incident logs (24 months), vendor contracts	Secondary evidence for requirements and controls
Expert panel	9	Purposive: 3 higher-ed IT security leaders, 3 governance and risk professionals, 3 AI or academic experts; 5 or more years' experience; no reporting line to you	6–10 experts are standard for CVI (Lynn, 1986; Polit and Beck, 2006)
Users	15	5 unit heads, 5 IT staff, 5 data owners who apply the toolkit	SUS gives usable estimates with about 12–14 users (Tullis and Stetson, 2004)
AI systems for demonstration	3	Chatbot, analytics dashboard, research assistant, chosen for varied vendor dependence and data sensitivity	Enough variety without overloading the evaluation

Do

Choose evaluators by expertise and role, and state inclusion criteria (for example, 5 or more years).
Pick demonstration systems that vary in vendor dependence and data sensitivity.
Give a reason for every sample size.

Don't

Use colleagues who will not challenge the artefact.
Evaluate only on the system you designed against.
Use fewer than 6 experts for CVI.

Mistakes students make here

Student writes: Experts are the student's colleagues and friends.
Why it fails: They will not challenge the artefact. The evaluation looks rigorous but tells nothing.
Fix: Set inclusion criteria (5 or more years, no reporting line) and recruit from outside the team.

Student writes: "Nine experts because that is what my friend used."
Why it fails: No reason given. Examiners will ask.
Fix: Cite the CVI convention (6–10 experts) and the SUS convention (about 12–14 users).

Student writes: Evaluating only on the system the artefact was designed against.
Why it fails: It shows the artefact fits one case, not that it works.
Fix: Choose systems that vary in vendor dependence and data sensitivity.

SMART check for this step

S: SpecificEach group has a role, a criterion and a count.
M: Measurable12 interviews, 9 experts, 15 users, 3 systems.
A: AchievableAccess agreed by PSU gatekeepers.
R: RealisticSizes match published minimums for CVI (6–10) and SUS (12–14).
T: Time-boundRecruited within 2 months of t0.

In-class check

Each student writes three lines of inclusion criteria for their expert panel. Then ask: which of your friends would be excluded by these criteria?

Discuss: why can you not run a t-test on three systems? (n is too small. Report descriptively.)

Chapter 3: Research design and methods · Step 16 of 23 · about 10 min

3.4Instruments and ethics

Teach this

The instrument is where propositions become measurable. Use neutral wording, standard scales where they exist, and get IRB clearance before any data. Students often underestimate how long IRB takes.

Worked example: PSU

Requirements interview guide (10 open questions). Sample: Which of PSU's AI systems hold data you would least want exposed, and who is accountable if it leaks?
Expert rating form: each control rated 1–4 for relevance and for clarity. Sample: Control C14, "The vendor must notify PSU of an AI-related breach within 72 hours", is relevant to AI security governance at PSU (1 = not relevant, 4 = highly relevant).
System Usability Scale (10 standard items, 5-point; Brooke, 1996), with "system" changed to "toolkit". Scored 0–100.
Utility item: The toolkit helps me prioritise AI security investment (5-point).
Risk-identification log: for each system, the distinct risks found by the earlier IT review and by the toolkit; the CISO adjudicates duplicates.

Ethics and IRB

IRB clearance for every instrument, including expert forms, before Cycle 1.
Informed consent; experts and users identified by code only.
Vulnerability details go to the CISO only; the dissertation reports aggregate findings and anonymises system names where needed.
Data stored encrypted for the period the IRB sets; GenAI use logged.

Do

Use standard instruments where they exist (SUS) and cite them.
Rate each control on relevance and clarity separately.
Get IRB clearance for all instruments, including the expert forms.

Don't

Ask experts leading questions ("Do you agree this excellent control is relevant?").
Publish vulnerability details from the demonstration.
Record employee names in incident logs.

Mistakes students make here

Student writes: "Do you agree that this excellent control is relevant?"
Why it fails: Leading item. It pushes the expert towards agreement and invalidates the CVI.
Fix: "Rate the relevance of this control from 1 (not relevant) to 4 (highly relevant)."

Student writes: Starting the expert workshop before IRB clearance.
Why it fails: Data collected without clearance can be thrown out, and it may end the candidacy.
Fix: No contact with participants until the instrument is cleared.

Student writes: Inventing a new usability scale instead of using SUS.
Why it fails: There is no validation, so the score cannot be compared with any benchmark.
Fix: Use SUS as published, changing only the word "system" to "toolkit", and cite Brooke (1996).

SMART check for this step

S: SpecificOne instrument per evaluation criterion.
M: MeasurablePilot the expert form with 2 people; no item flagged as confusing.
A: AchievableStandard SUS wording, with "toolkit" replacing "system".
R: RealisticEach form takes 30 minutes or less.
T: Time-boundIRB cleared before Cycle 1.

In-class check

Give the class one leading item and ask them to rewrite it neutrally in two minutes. Compare versions.

Ask: which of your instruments needs IRB clearance? (All of them, including the expert rating form.)

Chapter 3: Research design and methods · Step 17 of 23 · about 6 min

3.5Data collection plan

Teach this

Fieldwork is project management: dates, targets, logs and protected data. In DSR the log matters twice, because every design change must be traceable to its reason.

Worked example: PSU

When	Activity	Objective
Months 1–2	Requirements interviews and document review	RO1
Months 3–4	Build prototype v1	RO2
Months 4–5	Cycle 1: expert review (two CVI rounds); revise to v2	RO2, RO3
Months 6–7	Demonstrate on three AI systems; unit heads apply the toolkit and log risks	RO3
Months 7–8	Cycle 2: SUS, utility item, risk-log adjudication, final CVI confirmation	RO3

Do

Keep a fieldwork log: dates, response rates, problems, decisions.
Protect anonymity and store data encrypted, with backups.
Get an IRB amendment before changing any instrument.

Don't

Change instruments midway without approval.
Pressure people to respond.
Publish vulnerability details that could help an attacker.

Mistakes students make here

Student writes: No log kept during the cycles.
Why it fails: The student cannot say why the design changed, so the rigour cycle is missing.
Fix: Log each change with its reason and the evidence behind it.

Student writes: Adding a question to the form halfway through.
Why it fails: It changes the instrument and breaks the IRB approval.
Fix: Submit an amendment before any change.

Student writes: A timeline that ignores IRB and expert availability.
Why it fails: Delays cascade, and the student misses Milestone 2.
Fix: Plan backwards from the milestone, and mark IRB as a dependency.

SMART check for this step

S: SpecificInstrument, channel and reminders are defined.
M: MeasurableProgress tracked weekly against target.
A: AchievableGatekeeper support for distribution is confirmed.
R: RealisticA window of 4–6 weeks per phase.
T: Time-boundClosing date set before you begin.

In-class check

Have students draw their timeline backwards from Milestone 2 on a sheet of paper. Mark where IRB clearance sits and ask: what happens to every later date if it slips by four weeks?

Chapter 3: Research design and methods · Step 18 of 23 · about 12 min

3.6Statistical methods and evaluation metrics

Teach this

Decide the test and the threshold before you see any data, and say what result counts as failure. In DSR most metrics are simple (CVI, SUS, counts), but they still need decision rules.

Worked example: PSU

Question or proposition	Data	Method	Decision rule
RQ1 Requirements	Interview transcripts, documents	Reflexive thematic analysis (Braun and Clarke, 2006)	Keep a requirement if raised by 3 or more participants from 2 or more groups; list agreed by at least 80% in validation
DP1 Accuracy	Expert ratings (n = 9, 4-point)	I-CVI, S-CVI/Ave, Kendall's W	I-CVI of 0.78 or above (7 of 9 rate 3–4); S-CVI/Ave of 0.90 or above; W of 0.6 or above. Revise or drop items below threshold
DP2 Usability	SUS (n = 15)	Mean, SD, 95% CI, one-sample t-test against 68	Mean of 68 or above
DP3 Effectiveness	Risk logs for 3 systems	Count of distinct validated risks; percentage uplift	Mean uplift of 30% or more. Descriptive only, because n = 3
DP4 Utility	Utility item (n = 15)	Proportion rating 4–5 with Wilson 95% CI	80% or more
Expert and user comments	Free text	Thematic coding	Every design change is logged and linked to a proposition

Do

State the decision rule for each proposition before you evaluate.
Report failures and the design change each one caused.
Use descriptive statistics where n is too small for inference (3 systems).

Don't

Run inferential tests on 3 systems.
Drop controls that experts rate low without recording why.
Report a mean SUS score without its spread.

Mistakes students make here

Student writes: Running a t-test on three systems.
Why it fails: With n = 3, inference is meaningless.
Fix: Report the counts and the percentage uplift, and say the result is descriptive.

Student writes: "SUS score of 74%."
Why it fails: SUS is scored 0–100 but is not a percentage. The benchmark for average is 68.
Fix: Report the mean, SD, CI and the benchmark, and interpret: good, not excellent.

Student writes: Dropping low-rated controls without a record.
Why it fails: Hides the evidence that shaped the design.
Fix: Keep the rating table, record every drop and give the reason.

SMART check for this step

S: SpecificOne metric per proposition.
M: MeasurableThresholds: I-CVI 0.78, S-CVI/Ave 0.90, W 0.6, SUS 68, 80% utility.
A: AchievableSpreadsheet plus SPSS or R skills confirmed.
R: RealisticDoable by one researcher.
T: Time-boundAnalysis finished within 2 weeks of each cycle.

In-class check

Exercise: SUS with n = 15, mean 74.2 and SD 9.8. Compute the 95% confidence interval. Answer: SE = 9.8 / √15 = 2.53; t(14) = 2.145; CI = 74.2 ± 5.4 = [68.8, 79.6].

Ask: the lower bound is barely above 68. What does that tell you about how confident to be?

Chapter 4: Results and findings · Step 19 of 23 · about 10 min

4.1Results and findings (illustrative)

Teach this

Restate the proposition, show the evidence, state the decision. Failures and revisions are findings, and showing them is what makes a DSR study credible.

Worked example: PSU

Illustrative results. Replace with your own. RQ1: six requirements were retained (REQ1–REQ6 in section 1.8) after 12 interviews; 10 of 12 participants (83%) agreed the final list.

Proposition	Evidence	Decision
DP1 Accuracy	Cycle 1: 30 controls; 26 had I-CVI of 0.78 or above; 4 unclear vendor-notice controls were rewritten. S-CVI/Ave rose from 0.88 to 0.92 after revision; Kendall's W = 0.68	Supported after revision
DP2 Usability	SUS mean 74.2 (SD 9.8), 95% CI [68.8, 79.6]; t(14) = 2.45, one-sided p = .014; mean completion time 1 h 40 min	Supported. Above the 68 average but below the 80.3 top-quartile mark, so "good", not "excellent"
DP3 Effectiveness	Risks found, old vs toolkit: chatbot 6 vs 9; analytics 6 vs 8; research assistant 8 vs 11. Total 20 vs 28, an uplift of 40%	Supported (descriptive only)
DP4 Utility	13 of 15 users (87%) rated 4 or 5; Wilson 95% CI [62%, 96%]	Supported on the point estimate. The CI is wide, so treat it cautiously

Interpretation: the artefact met all four propositions and DO4 (completion under 2 hours). The Cycle 1 failure (unclear vendor-notice wording) shows why early expert review pays off. Compare against the literature: the usability score is consistent with tools that are light by design, and the uplift supports the claim that structure finds risks unstructured reviews miss. State the limits: three systems, one institution, self-reported utility.

Do

Restate each design proposition, then the evidence, then the decision.
Show what failed in Cycle 1 and how the design changed.
Interpret the results against the literature.

Don't

Report only the successful cycle.
Overstate a SUS of 74. It is good, not excellent.
Mix recommendations into the results.

Mistakes students make here

Student writes: Reporting only the final, successful cycle.
Why it fails: Hides what failed and why. It looks like the artefact worked first time, which examiners do not believe.
Fix: Show Cycle 1 failures and the design changes they caused.

Student writes: Mixing recommendations into the results.
Why it fails: Results describe evidence. Recommendations belong in Chapter 5.
Fix: Keep Chapter 4 to evidence and interpretation.

Student writes: Calling a SUS of 74 "excellent".
Why it fails: The benchmark for excellent is above 80. Overstating loses credibility.
Fix: Say "above average" and quote the benchmark.

SMART check for this step

S: SpecificOne results row per proposition.
M: MeasurableEach decision quotes its metric and threshold.
A: AchievableOnly data from approved evaluations.
R: RealisticThe chapter fits the word budget.
T: Time-boundDrafted within 6 weeks of finishing Cycle 2.

In-class check

Give students the raw results (I-CVI table, SUS mean and CI, risk counts). They draft the decision table for DP1–DP4 in ten minutes. Compare: which decision was hardest, and why?

Chapter 5: Conclusions, recommendations and further research · Step 20 of 23 · about 6 min

5.1Conclusions

Teach this

Conclusions answer the objectives and contain nothing new. Each one should point to the finding it rests on and the limit it carries.

Worked example: PSU

The artefact met its four design propositions: experts judged it accurate after one revision, unit heads found it usable in under two hours, it found 40% more risks than the earlier review, and most users found it useful for prioritising investment. The main scholarly output is six design principles:

DPr1 Give every AI system a named accountable owner.
DPr2 Assess data, model, deployment and operation together.
DPr3 Make vendor duties explicit and checkable.
DPr4 Map every control to a recognised standard so audits can reuse it.
DPr5 Keep assessment light enough for unit heads.
DPr6 Score all systems on the same scale so they can be compared.

Not shown: whether the artefact reduces incidents, or whether it works outside PSU.

Do

Link each conclusion to an objective, a hypothesis or proposition, and the literature.
State what the evidence supports and what it does not.
Say what changed in your understanding.

Don't

Introduce new data or new literature.
Claim broad generalisation from one case.
Repeat results without interpreting them.

Mistakes students make here

Student writes: Introducing a new source or new number in the conclusion.
Why it fails: Conclusions summarise. New evidence belongs in the earlier chapters.
Fix: Move it back, or delete it.

Student writes: "The artefact eliminates AI security risk at PSU."
Why it fails: Overclaims. The study never measured incidents.
Fix: State what the evaluation showed: accuracy, usability, risks found.

Student writes: Repeating the results table without interpretation.
Why it fails: Readers already saw it. A conclusion says what it means.
Fix: For each objective, write one sentence of meaning and one of limit.

SMART check for this step

S: SpecificOne conclusion per objective.
M: MeasurableEach cites the statistic or finding it rests on.
A: AchievableWithin what the data support.
R: RealisticScaled to a single case.
T: Time-boundDrafted within 4 weeks of finishing Chapter 4.

In-class check

Match each conclusion to an objective on the board. Any objective with no conclusion, or conclusion with no objective, needs work.

Chapter 5: Conclusions, recommendations and further research · Step 21 of 23 · about 10 min

5.2SMART recommendations

Teach this

Every recommendation is a link in the chain from problem to proposition to conclusion to recommendation, and it must pass all five SMART tests. The R in your guide is Realistic. Students often write it as Relevant.

Worked example: PSU

Chain shown in each row: Problem → Design proposition → Conclusion → Recommendation. The conclusions are conditional on your real results.

Chain	Recommendation	Specific	Measurable	Achievable	Realistic	Time-bound
SP2 no artefact → DP1 → experts judged levels accurate (S-CVI/Ave 0.92)	R1 Adopt the maturity model as PSU's AI onboarding assessment	CISO requires an assessment before any new AI go-live	100% of new systems assessed; level 3 needed to go live	Toolkit takes 2 hours or less per system	Uses the existing change-approval process	Within 4 months
SP1 undocumented requirements → DP1 → ownership requirement confirmed	R2 Name an owner for every AI system	Provost's office assigns an owner and a RACI	100% of production systems have a named owner	Within existing roles	One-page RACI template	Within 3 months
SP3 no evidence → DP2, DP3 → SUS 74; 40% more risks found	R3 Review the artefact yearly	Governance committee reviews controls annually	Annual report; 90% or more of controls re-checked	Two-day workshop	Fits the audit calendar	First review in 12 months

Do

Keep the chain: Problem → Hypothetical solution or proposition → Conclusion → Recommendation.
Check each recommendation on S, M, A, R and T (R means Realistic in your guide).
Name an owner and a metric.

Don't

Recommend anything your data does not support.
Write generic advice ("improve awareness").
Ignore budget and staff capacity.

Mistakes students make here

Student writes: "PSU should improve AI security awareness."
Why it fails: No owner, metric or deadline. Nobody can tell when it is done.
Fix: "The CISO will assess 100% of new AI systems with the toolkit within 4 months."

Student writes: A recommendation with no finding behind it.
Why it fails: It is an opinion, not evidence-based, and examiners will ask where it came from.
Fix: Cite the proposition and the result that support it.

Student writes: Ignoring budget and staff capacity.
Why it fails: The recommendation is not Realistic, so PSU will ignore it.
Fix: Use existing roles and tools, and say so.

SMART check for this step

S: SpecificOne action, one owner.
M: MeasurableA metric and a target.
A: AchievableWithin the owner's authority.
R: RealisticWithin budget and staff capacity.
T: Time-boundA deadline counted from adoption.

In-class check

Five-minute rewrite: take the vague recommendation "improve AI awareness" and make it SMART. Compare versions and point out which of S, M, A, R or T is weakest.

Chapter 5: Conclusions, recommendations and further research · Step 22 of 23 · about 4 min

5.3Areas for further research

Teach this

Further research grows out of the student's own limitations. Each item should name the limitation, the design that would answer it and what result would count as an answer.

Worked example: PSU

Limitation	Further research	Design	Horizon
Single institution	Evaluate the artefact at two other universities	Multi-site DSR evaluation with the same metrics	12–18 months
8-month evaluation; no incident data	Test whether adoption reduces incidents	Before-and-after design with 12 months of incident tracking	12–24 months
Self-assessment can be gamed	Compare self-assessment with independent audit	Validity study on 10 systems	12 months
15 users	Larger usability study	SUS plus task timing with 40 or more users	6–12 months

Do

Base each item on one of your own limitations or unanswered findings.
Say which design would answer it.
Keep to 3–5 items.

Don't

Write "more research is needed" with no direction.
List topics unrelated to your findings.
Promise to do it all yourself.

Mistakes students make here

Student writes: "More research is needed on AI security."
Why it fails: No direction. It could end any paper.
Fix: Name the limitation and the design.

Student writes: Suggesting topics unrelated to the findings.
Why it fails: It reads as a wish list, not a research agenda.
Fix: Derive each item from a limitation.

Student writes: A list of ten items.
Why it fails: Nobody will act on ten. Length signals weak prioritising.
Fix: Keep three to five.

SMART check for this step

S: SpecificEach item names a limitation and a design.
M: MeasurableEach states what result would count as an answer.
A: AchievableFeasible for a future research team.
R: RealisticWithin reach of one funded project.
T: Time-boundSuggests a horizon, for example 12–24 months.

In-class check

Each student turns one of their own limitations into a further-research item with a design and a horizon (for example 12–24 months).

Before you submit · Step 23 of 23 · about 8 min

6Traceability and submission check

Teach this

The traceability table proves the study hangs together. Every sub-problem has an objective, question, proposition, data, test and recommendation. An empty cell is a defect. Run it before every milestone.

Worked example: PSU

Sub-problem	Objective	Question	Proposition	Data	Test	Recommendation
SP1	RO1	RQ1	Requirements REQ1–REQ6	12 interviews, documents	Thematic analysis; 80% agreement	R2 owner per system
SP2	RO2	RQ2	DP1	9 experts	I-CVI of 0.78 or above; Kendall's W	R1 onboarding assessment
SP3	RO3	RQ3	DP2, DP3, DP4	15 users; 3 systems	SUS of 68 or above; risk uplift; utility of 80% or more	R3 annual review

If any cell is empty, the design is broken. Fix it before you write more.

Do

Check that every sub-problem has an objective, question, hypothesis or proposition, data, test and recommendation.
Run the check before every milestone.
Fix empty cells before you write more.

Don't

Add a recommendation with no finding behind it.
Leave a hypothesis with no test.
Change one link without updating the others.

Mistakes students make here

Student writes: A recommendation with no problem behind it.
Why it fails: It floats free of the evidence.
Fix: Trace it back to a sub-problem, or remove it.

Student writes: A proposition with no test.
Why it fails: It cannot be decided, so it cannot be a finding.
Fix: Name the metric and the threshold.

Student writes: Changing an objective without updating the questions and tests.
Why it fails: The chapters drift apart, and the viva finds it.
Fix: Update all links whenever one changes.

SMART check for this step

S: SpecificEvery row is complete.
M: MeasurableZero empty cells.
A: AchievableBuilt from your own chapters, not from this page.
R: RealisticTakes under an hour.
T: Time-boundDone before each milestone.

In-class check

Peer audit: students swap traceability tables and mark every empty cell in red. Ten minutes. Each student then fixes the cells their partner found.

Anchor references

Starting points named in this walkthrough. Open each one and confirm the details before you cite it or teach from it.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In P. W. Jordan et al. (Eds.), Usability evaluation in industry. Taylor & Francis.
Gregor, S., & Hevner, A. R. (2013). Positioning and presenting design science research for maximum impact. MIS Quarterly, 37(2), 337–355.
Hevner, A. R. (2007). A three cycle view of design science research. Scandinavian Journal of Information Systems, 19(2), 87–92.
Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105.
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–385.
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734.
National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0).
Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45–77.
Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what's being reported? Research in Nursing & Health, 29(5), 489–497.
Tullis, T. S., & Stetson, J. N. (2004). A comparison of questionnaires for assessing website usability. Usability Professionals' Association Conference.
Venable, J., Pries-Heje, J., & Baskerville, R. (2016). FEDS: A framework for evaluation in design science research. European Journal of Information Systems, 25(1), 77–89.

Session	Steps	Teaching time
Session 1: Frame the study	1.1 to 1.6	About 47 min, plus discussion
Session 2: Build the argument	1.7 to 2.2	About 55 min, plus discussion
Session 3: Design the method	3.1 to 3.6	About 56 min, plus discussion
Session 4: Report and act	4.1 to 6	About 38 min, plus discussion

Jump to a step

Chapter 1: Introduction
1.1Background
1.2Problem statement
1.3Sub-problems
1.4Purpose
1.5Research objectives (SMART)
1.6Research questions
1.7Hypotheses, propositions and integration expectations
1.8Conceptual framework
1.9Scope, assumptions and limitations
1.10Contribution to praxis
Chapter 2: Review of literature
2.1Literature themes mapped to research questions
2.2Research and practitioner gap
Chapter 3: Research design and methods
3.1Research design
3.2Justification of the approach
3.3Data and sample
3.4Instruments and ethics
3.5Data collection plan
3.6Statistical methods and evaluation metrics
Chapter 4: Results and findings
4.1Results and findings (illustrative)
Chapter 5: Conclusions, recommendations and further research
5.1Conclusions
5.2SMART recommendations
5.3Areas for further research
Before you submit
6Traceability and submission check

Chapter 1: Introduction · Step 1 of 23 · about 8 min

1.1Background

Teach this

In a mixed-methods study the background must show why numbers alone and stories alone are each incomplete. That one idea justifies the whole design.

Worked example: PSU

These systems add attack surfaces that conventional IT security only partly covers: prompt injection, leakage of prompt or training data, and dependence on vendors (cite). India's Digital Personal Data Protection Act 2023 raises PSU's duties as a data fiduciary (cite).

Inside PSU, concern is voiced but not measured. Incidents are rarely reported, and students, faculty, administrators and IT staff seem to see the risk differently. Counting how widespread concern is and explaining why it exists are two different jobs, so the study needs both numbers and explanations. PSU here stands for a public university, used as an illustration.

Do

Move from the global trend to PSU using dated, credible evidence: regulator reports, peer-reviewed studies and PSU documents.
Frame it as a business problem with scholarly relevance. A DBA needs both.
Confirm access to PSU data, people and documents before you commit to the topic.

Don't

Write a general essay on AI in the world.
Pick a topic whose data PSU may not release.
Use vendor blogs or press releases as core evidence.

Mistakes students make here

Student writes: "Concern about AI security is growing everywhere."
Why it fails: Vague and unreferenced. It says nothing about PSU or why two methods are needed.
Fix: Name PSU's five systems, the fact that incidents are rarely reported and that groups seem to disagree.

Student writes: A background that never explains why both methods are needed.
Why it fails: Examiners then read the interviews as bolted on.
Fix: End the section with the need for extent (how many) and reasons (why).

Student writes: Cites only survey studies, or only qualitative case reports.
Why it fails: It hides the gap the design is meant to fill.
Fix: Show one of each kind and say what each can and cannot tell PSU.

SMART check for this step

S: SpecificNames PSU, the AI systems in use and the security concern.
M: MeasurableAt least 10 credible sources, half from the last five years, plus one PSU document showing the adoption.
A: AchievableWritten access confirmation from a PSU gatekeeper.
R: RealisticFits the 8–10 page topic proposal.
T: Time-boundDrafted before Immersion 1.

In-class check

Ask each student to write one question about PSU that a survey can answer and one that it cannot. Read a few aloud. This is the seed of the mixed design.

Chapter 1: Introduction · Step 2 of 23 · about 10 min

1.2Problem statement

Teach this

The problem statement should contain two kinds of unknown: how much and why. If it has only one, the second method has no reason to exist.

Worked example: PSU

PSU does not know how widespread AI security concern is across its stakeholders, or why it exists, so it cannot decide where to act.

Practitioner gap: leaders decide on anecdote, so training and controls may go to the wrong group. Research gap: studies tend to measure perceived risk or to explain it, rarely both, and rarely across all four stakeholder groups in one institution.

Do

Write one paragraph: what is happening, what should be happening, who is affected, what is unknown.
Split the gap into a practitioner gap and a research gap.
Keep the problem solvable with data you can reach.

Don't

Name a technology fix inside the problem ("PSU needs an AI firewall").
Write moral or vague statements ("AI is dangerous").
Choose a problem too large for the time you have.

Mistakes students make here

Student writes: "PSU does not know the level of AI security concern."
Why it fails: Only a how-much problem. The interviews have nothing to explain.
Fix: Add the why: "...or why it exists, so it cannot decide where to act."

Student writes: "PSU needs a survey of its stakeholders."
Why it fails: Names the method instead of the unknown.
Fix: State what is not known, not how it will be found.

Student writes: Uses "stakeholders" without naming who they are.
Why it fails: The reader cannot see the sample or the comparison the study depends on.
Fix: Name the four groups: students, faculty, administrators, IT staff.

SMART check for this step

S: SpecificOne population, one setting, one paragraph.
M: MeasurableThe gap is evidenced by at least 3 sources and one PSU fact.
A: AchievableThe data needed to address it is accessible.
R: RealisticSolvable at doctoral scale.
T: Time-boundSigned off by your chair before Milestone 1.

In-class check

Circle every "how much" word and every "why" word in the problem statement. There must be at least one of each. If one is missing, rewrite it now.

Chapter 1: Introduction · Step 3 of 23 · about 6 min

1.3Sub-problems

Teach this

Sub-problems split the problem into extent, reasons and action. Each needs its own data source, otherwise the study has fewer moving parts than it claims.

Worked example: PSU

SP1 The extent of perceived AI security risk and trust across stakeholder groups is unmeasured.
SP2 The reasons behind those perceptions are unexplained.
SP3 PSU has no evidence-based basis for choosing where to act first.

Do

Make each sub-problem a component of the main problem.
Keep them to 3–5, each mapped to one objective.
Phrase each as a gap, not as a solution.

Don't

List overlapping sub-problems that need the same data.
Add issues you will not research (cost, bias, ethics).
Phrase a sub-problem as a solution.

Mistakes students make here

Student writes: Three sub-problems, all about measuring different things.
Why it fails: Nothing calls for interviews. The design is quantitative in disguise.
Fix: One sub-problem for extent, one for reasons, one for action.

Student writes: SP2 restates SP1 with "and why" added.
Why it fails: Two sub-problems, one dataset. The mapping to objectives will not hold.
Fix: Give SP2 its own data: interviews.

Student writes: No sub-problem about action, so the integration has no purpose.
Why it fails: The study ends with findings and no decision.
Fix: Add SP3: no basis for choosing where to act.

SMART check for this step

S: SpecificEach states one issue.
M: MeasurableEach traces to exactly one research objective.
A: AchievableEach has an accessible data source.
R: Realistic3–5 in total.
T: Time-boundFixed before you write the objectives.

In-class check

On the board draw three columns: SP, RO, RQ. Students fill them from their drafts. Any orphan or duplicate must be fixed before the next step.

Chapter 1: Introduction · Step 4 of 23 · about 5 min

1.4Purpose

Teach this

The purpose statement names the design, the order and the priority: explanatory sequential, survey first, quantitative priority. "Mixed methods" on its own is not a design.

Worked example: PSU

The purpose of this explanatory sequential mixed-methods study is to examine perceived AI security risk and trust among PSU students, faculty, administrators and IT staff (quantitative), and then to explain the patterns through stakeholder interviews (qualitative).

The study produces a measured picture, an explained picture and a ranked list of where PSU should act.

Do

Use one sentence with a verb, the design, the population and the setting.
Match the wording to your design (explanatory, quantitative, design science).
State what the study will produce.

Don't

Write "to explore" when you will test hypotheses.
Promise outcomes the design cannot deliver ("to eliminate risk").
Leave the setting or population vague.

Mistakes students make here

Student writes: "The purpose is to use a mixed-methods approach."
Why it fails: It names no design, no order and no population.
Fix: "...an explanatory sequential mixed-methods study... quantitative first, then qualitative to explain."

Student writes: Calling an explanatory sequential design "triangulation".
Why it fails: Triangulation means two strands checking the same thing. Here the second strand explains the first.
Fix: Use "explain" or "expand" and reserve triangulation for convergent designs.

Student writes: Writing two purposes: one for the survey, one for the interviews.
Why it fails: It reads as two studies stapled together.
Fix: One purpose with two linked strands.

SMART check for this step

S: SpecificVerb, design, population and setting all named.
M: MeasurableThe output can be checked: a model, a test result or a framework.
A: AchievableConsistent with the data you can reach.
R: RealisticAchievable within the programme timeline.
T: Time-boundApproved by your chair at Milestone 1.

In-class check

One-breath test: each student reads their purpose in a single breath. Then ask: which word tells the reader the survey comes first?

Chapter 1: Introduction · Step 5 of 23 · about 10 min

1.5Research objectives (SMART)

Teach this

Each objective gets a verb, a number and a deadline from t0, and the deadlines must respect the sequence: the interviews cannot finish before the survey has been analysed.

Worked example: PSU

Objective (SP, RQ)	Specific	Measurable	Achievable	Realistic	Time-bound
RO1 Measure (SP1, RQ1)	Perceived risk and trust across four stakeholder groups	400 valid responses (at least 380 required); alpha of 0.70 or above	Stratified list from the PSU registry	Survey takes 12 minutes or less	t0 + 4 months
RO2 Explain (SP2, RQ2)	Reasons behind the survey patterns	12–15 interviews, until no new codes appear in the last 3	Participants chosen from survey volunteers	One-hour interviews	t0 + 6 months
RO3 Integrate (SP3, RQ3)	Link both strands and rank where to act	1 joint display; at least 4 meta-inferences; ranked action list	Uses survey and interview outputs	No extra data collection	t0 + 8 months

Do

Start each objective with a measurable verb: identify, measure, test, design, evaluate.
Count deadlines from a fixed event such as IRB clearance (t0), not from a calendar date you cannot control.
Keep objective, research question and sub-problem one-to-one.

Don't

Use "understand", "explore" or "look into" for objectives you will test.
Promise outcomes the design cannot deliver.
Add an objective that has no data behind it.

Mistakes students make here

Student writes: "RO2: To gain insight into why stakeholders feel this way."
Why it fails: "Gain insight" cannot be measured or checked.
Fix: "Conduct 12–15 interviews until the last 3 add no new codes, by t0 + 6 months."

Student writes: "RO3: To integrate the findings."
Why it fails: No product, no number. Students then integrate by writing two paragraphs.
Fix: "Produce one joint display with at least 4 meta-inferences by t0 + 8 months."

Student writes: Interviews scheduled to end before the survey is analysed.
Why it fails: In an explanatory sequential design the survey results choose the interviewees.
Fix: Check the dependency: RO2 cannot finish before RO1.

SMART check for this step

S: SpecificOne verb, one outcome, one population.
M: MeasurableA number, a threshold or a named test.
A: AchievableData and access already confirmed.
R: RealisticFits your resources and the programme timeline.
T: Time-boundDeadline counted from t0.

In-class check

Students underline the verb, number and deadline in each objective. Then ask: which objective depends on another? Draw the arrow.

Chapter 1: Introduction · Step 6 of 23 · about 8 min

1.6Research questions

Teach this

A mixed-methods study needs three kinds of question: quantitative, qualitative and a mixed question about how the strands relate. The third is the one most students forget.

Worked example: PSU

Research question	Type	Objective	Answered by
RQ1 To what extent do perceived risk and trust differ across PSU stakeholder groups?	Quantitative	RO1	Survey
RQ2 Why do stakeholders hold these views?	Qualitative	RO2	Interviews
RQ3 How do the interview explanations elaborate the survey patterns?	Mixed	RO3	Joint display

Do

Ask open questions: what, how, to what extent.
Keep them to 3–5, one per objective.
Put question, objective and hypothesis in one traceability table.

Don't

Ask yes/no questions ("Is AI secure?").
Ask what your data cannot answer.
Mix two constructs in one question.

Mistakes students make here

Student writes: Only two questions: one about the survey, one about the interviews.
Why it fails: No question asks what integration adds, so the design has no reason to combine.
Fix: Add a mixed question: how do the interviews elaborate the survey patterns?

Student writes: "RQ3: What are stakeholders' views?"
Why it fails: A repeat of RQ1 and RQ2. It does not ask about integration.
Fix: "How do interview explanations elaborate the survey patterns?"

Student writes: "RQ2: How many stakeholders distrust the tools?"
Why it fails: A counting question given to the interview strand.
Fix: Ask why or how in the qualitative question.

SMART check for this step

S: SpecificOne construct or artefact per question.
M: MeasurableYou can name the data and test that answers it.
A: AchievableAnswerable with data from PSU.
R: RealisticAnswerable within your timeline.
T: Time-boundTied to the objective's deadline.

In-class check

Have students label each of their questions QUAN, qual or mixed. Ask: do all three labels appear? If not, what is missing?

Chapter 1: Introduction · Step 7 of 23 · about 12 min

1.7Hypotheses, propositions and integration expectations

Teach this

Hypotheses belong to the survey strand. Propositions belong to the interview strand. Integration expectations connect the two, including what to do if they disagree.

Worked example: PSU

Hypotheses (survey strand)

Hypothesis	Claim	Decision rule
H1	Perceived AI security risk has a negative effect on trust in PSU's AI tools	β negative, p below .05 after Holm correction, 95% CI excludes 0
H2	Trust has a positive effect on continued use intention	β positive, p below .05, 95% CI excludes 0
H3	Perceived governance maturity is associated with lower perceived risk	β negative, p below .05, 95% CI excludes 0
H4	Perceived risk differs across the four stakeholder groups	Welch ANOVA p below .05 and η² reported

Propositions (interview strand)

P1 Fragmented accountability between IT, academic units and vendors explains unmanaged risk.
P2 Lack of transparency about vendor-run AI drives distrust.

Integration expectations

Convergence: groups closest to the systems (IT staff) report the highest perceived risk.
Expansion: interviews explain why administrators' trust is lowest.
Watch for discordance, where numbers and stories disagree.

Do

Write hypotheses for the survey strand and propositions for the interview strand, and say how they will be integrated.
State direction and a decision rule for each hypothesis (p, effect size, CI).
State an integration expectation: confirm, expand or explain.

Don't

Write hypotheses for the interview strand. Qualitative strands use propositions or questions.
Say "prove". Say "supported" or "not supported".
Change hypotheses after you see the survey results.

Mistakes students make here

Student writes: "H4: Interviewees will say trust is low."
Why it fails: A hypothesis about interviews. Qualitative strands use propositions or questions.
Fix: Write a proposition: fragmented accountability explains unmanaged risk.

Student writes: "H1: Perceived risk affects trust."
Why it fails: No direction and no decision rule.
Fix: "H1: Perceived risk has a negative effect on trust: β negative, 95% CI excludes 0."

Student writes: No plan for what to do if the strands disagree.
Why it fails: When the survey and interviews disagree, the student has no way to interpret it.
Fix: State integration expectations and name discordance as a possible result.

SMART check for this step

S: SpecificOne relationship per hypothesis; one claim per proposition.
M: MeasurableA test and decision rule named for each (p, effect size, theme evidence).
A: AchievableTestable with the instrument and interview guide you have.
R: RealisticGrounded in theory (trust, acceptance) and prior studies.
T: Time-boundFixed before the survey opens.

In-class check

For each hypothesis ask: what result would surprise you, and what would the interviews need to explain then?

Chapter 1: Introduction · Step 8 of 23 · about 12 min

1.8Conceptual framework

Teach this

The framework shows the sequence and the three integration points: connecting, building and interpreting. Each construct in the survey needs a matching probe in the interview guide.

Worked example: PSU

The survey strand tests one chain: perceived governance maturity (H3) → perceived risk → trust (H1) → continued use (H2), with stakeholder group as the comparison (H4). Every construct gets 3 or more survey items and one interview probe, so the two strands can be compared row by row.

Do

Show the sequence (survey, connect, interviews, integrate) and where integration happens.
Link each survey construct to at least one interview probe.
Say which strand has priority.

Don't

Draw two separate studies with no arrows between them.
Add constructs that only one strand measures.
Leave integration to the last chapter.

Mistakes students make here

Student writes: Two parallel boxes, "Survey" and "Interviews", with no arrows.
Why it fails: It looks like two studies. The design's logic is invisible.
Fix: Draw arrows for connecting, building and interpreting.

Student writes: Interviews probe topics the survey never measured.
Why it fails: The strands cannot be compared, so there is nothing to integrate.
Fix: Tie each construct to survey items and an interview probe.

Student writes: Leaving integration to the last chapter.
Why it fails: Integration then becomes a summary, not an analysis.
Fix: Plan the joint display now, and name its columns.

SMART check for this step

S: SpecificThree integration points named: connecting, building, interpreting.
M: MeasurableEvery construct has 3 or more survey items and one interview probe.
A: AchievableFeasible within 8 months.
R: RealisticSimple enough to explain in 2 minutes.
T: Time-boundFrozen before instrument design.

In-class check

Pick one construct, for example perceived governance maturity. On the board write its survey items on the left and its interview probe on the right. If a student cannot fill both sides, the strands do not connect.

Chapter 1: Introduction · Step 9 of 23 · about 6 min

1.9Scope, assumptions and limitations

Teach this

Each strand has its own limitations. Name the limitation of the sequence too: the interview sample depends on the survey, so any survey bias carries forward.

Worked example: PSU

In scope

AI systems in production at PSU for at least 6 months.
Four stakeholder groups: students, faculty, administrators, IT staff.
Security perceptions: confidentiality, integrity, availability, privacy.

Out of scope

Technical testing of the systems; AI ethics and bias except where they affect security.

Assumptions

PSU grants access to the registry for sampling.
Respondents answer honestly and interviewees speak candidly.

Limitations and mitigations

Limitation	Mitigation
One institution	Use interviews to explain, not to generalise; describe PSU in enough detail for readers to judge transfer (Lincoln and Guba, 1985)
Self-reported perceptions	Triangulate with policies and vendor contracts
Cross-sectional survey	Say "is associated with", never "causes"
Volunteer bias in interviews	Select by survey pattern, not by enthusiasm

Do

Say what is in, what is out and why.
Give each limitation a mitigation.
Separate delimitations (your choices) from limitations (constraints).

Don't

Hide weaknesses. Examiners find them.
Apologise for a limitation. State it and mitigate it.
Let the scope grow during data collection.

Mistakes students make here

Student writes: "The study covers all stakeholders at all universities."
Why it fails: It covers one institution. The claim cannot be supported.
Fix: State PSU, four groups and systems live for at least 6 months.

Student writes: No mention that the interview sample depends on survey results.
Why it fails: Examiners will see the dependency and ask.
Fix: State it and say how selection is made by pattern, not by enthusiasm.

Student writes: Generalising interview themes statistically ("80% of staff think...").
Why it fails: Thirteen interviews cannot support percentages.
Fix: Use themes to explain, and use the survey to describe extent.

SMART check for this step

S: SpecificEach boundary is explicit: systems, groups, period.
M: MeasurableEach limitation lists a named mitigation.
A: AchievableThe scope matches your access.
R: RealisticThe scope fits the time and the word limit.
T: Time-boundLocked at Milestone 2 and changed only with your chair's approval.

In-class check

Each student writes one limitation of the survey strand and one of the interview strand, with a mitigation for each.

Chapter 1: Introduction · Step 10 of 23 · about 5 min

1.10Contribution to praxis

Teach this

Using mixed methods is not a contribution. The contribution is what integration told the field that neither strand could alone, such as explaining a null result.

Worked example: PSU

Theory: extends trust and acceptance research by adding stakeholder differences and governance perception, and shows how integration explains a null result (trust not predicting use).

Practice: a ranked list of where PSU should act first, and an interview-based account of why, which a survey alone would not give.

Do

State the theoretical contribution and the practical contribution separately.
Tie each contribution to the gap you named.
Say who will use the result at PSU and beyond.

Don't

Overclaim generalisation from one institution.
List contributions your results cannot support.
Confuse a recommendation with a contribution.

Mistakes students make here

Student writes: "The contribution is a mixed-methods approach to AI security."
Why it fails: A method is not a contribution. It is how the work was done.
Fix: State what integration revealed: for example, why trust did not predict use.

Student writes: "The results apply to all universities."
Why it fails: Overclaims from one institution and thirteen interviews.
Fix: Say the explanations may transfer and how to test that.

Student writes: No practical contribution.
Why it fails: A DBA study has to help someone act.
Fix: Offer a ranked list of where PSU should act first.

SMART check for this step

S: SpecificOne theoretical and one practical contribution, each named.
M: MeasurableEach is tied to a specific finding or artefact you can point to.
A: AchievableWithin what your design can support.
R: RealisticScaled to a single-institution study.
T: Time-boundConfirmed in Chapter 5 after the results.

In-class check

Ask: what did integration tell you that neither the survey nor the interviews could tell you alone? If the answer is "nothing", the design was not needed.

Chapter 2: Review of literature · Step 11 of 23 · about 15 min

2.1Literature themes mapped to research questions

Teach this

Review both kinds of evidence by theme. Each theme needs supporters, challengers and neutral voices, and one theme should cover integration in mixed-methods research itself.

Worked example: PSU

Theme (RQ)	Supports your propositions	Challenges	Neutral	Critical evaluation
Risk perception and trust in educational technology (RQ1)	Perceived risk lowers trust (Mayer et al., 1995)	Mandated use weakens the effect; privacy paradox (Norberg et al., 2007)	Effect varies with context	Mostly student samples and self-report
Stakeholder differences in risk perception (RQ1)	Role and expertise shape perceived risk	Institutional culture matters more than role	Differences are small and vary by risk type	Few studies compare all four groups in one institution
Explaining the gap: governance, accountability, culture (RQ2)	Fragmented accountability and weak vendor oversight explain unmanaged risk	Technical controls, not governance, explain incidents	Both matter	Mostly practitioner reports with little method detail
Integration in mixed methods research (RQ3)	Sequential designs explain unexpected numbers (Creswell and Plano Clark, 2018)	Integration is often superficial; strands are reported separately	Value depends on the question	Few security studies show a joint display (Fetters et al., 2013)

Add at least two named sources in each cell from your database search.

Do

Organise by theme and map each theme to one research question.
For each theme show authors who support, challenge and are neutral to your propositions.
Evaluate the studies (sample, method, context, bias) and end with the gap.

Don't

Write an annotated bibliography ("Smith said… Jones said…").
Cite only papers that agree with you.
Cite from memory or from AI output without opening the paper. AI tools can invent references.

Mistakes students make here

Student writes: Reviews only quantitative work on risk and trust.
Why it fails: The qualitative strand has no literature behind it, so RQ2 looks unmotivated.
Fix: Include qualitative studies on why people trust or distrust technology.

Student writes: No literature on integration.
Why it fails: RQ3 then has no support and no method for the joint display.
Fix: Cite Creswell and Plano Clark (2018) and Fetters et al. (2013).

Student writes: Cites sources that were never opened, including AI-generated references.
Why it fails: AI tools can invent references. Invented citations are an integrity breach.
Fix: Open every source and keep a search log.

SMART check for this step

S: SpecificEach theme answers one research question.
M: MeasurableAt least 2 sources per position (support, challenge, neutral) per theme; half from the last 5 years.
A: AchievableSources available through your library.
R: Realistic12–18 pages for the full review.
T: Time-boundSearch log and theme matrix complete before Milestone 1.

In-class check

Assign each student a theme. They find one quantitative and one qualitative source on it, and one line on how they agree or clash. Check three references live.

Chapter 2: Review of literature · Step 12 of 23 · about 5 min

2.2Research and practitioner gap

Teach this

A mixed-methods gap usually reads: studies measure X or explain X, but not both, and not for these groups. The gap should show why one method would not have been enough.

Worked example: PSU

Existing studies either measure perceived AI risk or explain it, and few compare students, faculty, administrators and IT staff inside one institution. PSU therefore cannot tell how widespread concern is, who holds it, or why. This study measures, explains and integrates.

Do

State the gap as what the literature cannot yet tell PSU.
Link it back to the problem statement.
Keep it to one paragraph.

Don't

Claim nobody has ever studied the topic. Say what is missing in your context.
Invent a gap by ignoring existing work.
Leave the gap disconnected from your research questions.

Mistakes students make here

Student writes: A gap that talks only about missing measurement.
Why it fails: It supports a survey-only study.
Fix: Include the missing explanation.

Student writes: "No mixed-methods study of AI security exists."
Why it fails: Absolute and easy to refute. It also makes the method the gap.
Fix: Say what is missing in this setting: comparisons across four groups within one institution.

Student writes: A gap disconnected from the design.
Why it fails: The design then looks like a preference, not a response to the gap.
Fix: Link each half of the gap to a strand.

SMART check for this step

S: SpecificOne gap, named precisely.
M: MeasurableSupported by at least 3 rows of the theme matrix.
A: AchievableAnswerable by your study.
R: RealisticSized to your design.
T: Time-boundWritten before Milestone 1.

In-class check

Each student completes: "Studies measure ____ or explain ____, but not both for ____." The class checks that each blank is specific.

Chapter 3: Research design and methods · Step 13 of 23 · about 12 min

3.1Research design

Teach this

Name the design, the order and the priority, and state the rule that links the strands. In an explanatory sequential design the survey results decide who is interviewed.

Worked example: PSU

Explanatory sequential design, quantitative priority (QUAN → qual). The survey runs first and decides who is interviewed. The interview guide is then written from the survey results, and a joint display combines both strands.

Phase	What you do	Output	Deadline
1 Survey	Field the survey to a stratified sample; analyse H1–H4	Measured picture (RQ1)	t0 + 4 months
2 Connect	Select 12–15 interviewees by group and trust level, from survey volunteers	Interview list	t0 + 4.5 months
3 Interviews	Write the guide from the results; interview to saturation	Themes and propositions (RQ2)	t0 + 6 months
4 Integrate	Build the joint display; classify meta-inferences	Ranked actions (RQ3)	t0 + 8 months

Do

Name the design (explanatory sequential, quantitative priority) and cite a mixed methods authority.
Say exactly how survey results decide who is interviewed.
Plan the joint display now.

Don't

Use mixed methods because it sounds impressive.
Collect interview data before you analyse the survey.
Treat the interviews as a few quotes to decorate numbers.

Mistakes students make here

Student writes: Calls it explanatory sequential but interviews first, or at the same time.
Why it fails: The name and the timeline disagree.
Fix: Survey first, analyse, then interview.

Student writes: No selection rule for the interviewees.
Why it fails: The strands are unconnected and the interviews look arbitrary.
Fix: Select by group and trust level after the survey, for maximum variation.

Student writes: The joint display is planned only when writing up.
Why it fails: The interviews then miss the questions that integration needs.
Fix: Design the joint display's columns before you write the guide.

SMART check for this step

S: SpecificDesign, priority and sequence are named.
M: MeasurableThe selection rule for interviewees is explicit.
A: AchievableSkills for both strands exist or are trained.
R: RealisticThe 8-month sequence is feasible.
T: Time-boundPhases are dated from t0.

In-class check

On the board draw the design: survey, connect, interviews, integrate. Mark the three integration points. Ask: which arrow is weakest in your study?

Chapter 3: Research design and methods · Step 14 of 23 · about 8 min

3.2Justification of the approach

Teach this

Justify each method by the question it answers: numbers for extent, interviews for reasons. Reject the obvious alternatives, and admit the cost of running two strands.

Worked example: PSU

Each research question needs its own method: numbers for extent (RQ1), interviews for reasons (RQ2), integration for meaning (RQ3). The philosophy is pragmatism: choose the method that answers the question, then connect them.

Alternative considered	Why it was rejected
Survey only	Shows how much and who, but cannot explain why
Interviews only	Explains reasons but cannot show how widespread they are
Concurrent design	Interviews could not be targeted at surprising survey patterns
Design science	Builds a tool. The study's aim is understanding and prioritising, not building

Do

Explain why each RQ needs its method: numbers for extent, interviews for reasons.
Reject alternatives (survey only, interviews only, concurrent design) with one reason each.
State your philosophy (pragmatism) and what it means for integration.

Don't

Claim triangulation when the strands answer different questions. Say expansion.
Ignore the extra time and cost of two strands.
Skip the integration rationale.

Mistakes students make here

Student writes: "Mixed methods gives the best of both worlds."
Why it fails: A slogan, not a justification. It says nothing about the questions.
Fix: Say which RQ needs which method.

Student writes: Claiming triangulation for a sequential design.
Why it fails: The strands answer different questions, so they expand rather than triangulate.
Fix: Say expansion or explanation.

Student writes: Ignoring the extra time and cost.
Why it fails: Examiners ask whether the student can finish both strands.
Fix: State the time plan and what would be cut if it slipped.

SMART check for this step

S: SpecificThree or more alternatives rejected with one reason each.
M: MeasurableEach reason cites a methods source.
A: AchievableBoth sets of skills exist or are trained.
R: RealisticFits one page.
T: Time-boundWritten before Milestone 1.

In-class check

Role play: one student is a quantitative purist ("just survey more people"), one a qualitative purist ("numbers cannot explain"). The defender has three minutes to justify the sequence. Swap roles.

Chapter 3: Research design and methods · Step 15 of 23 · about 8 min

3.3Data and sample

Teach this

Use a different sampling logic for each strand and connect them. Stratified random sampling suits the survey. Purposeful sampling by survey pattern suits the interviews.

Worked example: PSU

Strand	Population and frame	Sampling	n	Why this size
Survey	About 10,000 people: 8,000 students, 1,200 faculty, 600 administrators, 200 IT staff (PSU registry)	Stratified random with group minimums. Target: students 208, faculty 82, administrators 61, IT staff 61. About 580 invited	412 valid (target 400; at least 380 required)	Krejcie and Morgan (1970): about 370 for N of 10,000. Group minimums allow comparison. 45% extra for non-response
Interviews	Survey volunteers who consent to follow-up	Purposeful, maximum variation by group and trust level (high and low) after the survey	12–15	Stop when the last 3 interviews add no new codes (Guest et al., 2006)
Documents	Policies, vendor contracts, incident logs	Purposive	All relevant	Triangulates perceptions

Do

Use a different sampling logic for each strand: stratified random for the survey, purposeful for interviews.
Select interviewees using survey results (group by trust level).
Stop interviews at saturation and report how you judged it.

Don't

Use the same people in both strands and analyse them as if independent.
Interview only volunteers who agree with the survey story.
Set n without a rule.

Mistakes students make here

Student writes: Convenience sampling for both strands.
Why it fails: The survey cannot compare groups, and the interviews cannot explain patterns.
Fix: Stratify the survey; select interviewees by survey pattern.

Student writes: Interviewing only the volunteers who agree with the survey story.
Why it fails: Selection bias. The interviews then confirm what the student expected.
Fix: Choose maximum variation: high and low trust in each group.

Student writes: "Six interviews are enough."
Why it fails: No rule for saturation, so the claim cannot be judged.
Fix: State a rule: stop when the last 3 interviews add no new codes (Guest et al., 2006).

SMART check for this step

S: SpecificSurvey: 400 valid across four groups. Interviews: 12–15.
M: MeasurableSurvey: Krejcie and Morgan plus group minimums. Interviews: saturation rule.
A: AchievableVolunteers recruited through the survey.
R: Realistic70% response assumed; 580 invited.
T: Time-boundThe survey closes before interview selection starts.

In-class check

Exercise: 580 invited, 71% respond. How many valid responses? (580 × 0.71 ≈ 412.) Krejcie and Morgan give about 370 for a population of 10,000. Why still insist on at least 60 per group? (So that group comparisons are possible.)

Chapter 3: Research design and methods · Step 16 of 23 · about 10 min

3.4Instruments and ethics

Teach this

The two instruments must connect but not repeat. The interview guide is written after the survey so its probes can target the survey patterns.

Worked example: PSU

Construct	Items	Sample item (5-point agree scale)
Perceived AI security risk	5	I am concerned that my personal data could be exposed when I use PSU's AI tools.
Trust in PSU's AI tools	5	I believe PSU's AI tools handle my data responsibly.
Perceived governance maturity	4	PSU has clear rules about who is responsible for each AI tool.
Continued use intention	4	I intend to keep using PSU's AI tools next semester.
Controls and follow-up	4	Group, age band, frequency of AI use, willingness to be interviewed

Adapt items from validated trust and acceptance scales (Mayer et al., 1995; Venkatesh et al., 2003) and cite each source. Pilot with 30 people.

Interview guide (written after the survey)

The survey showed administrators trust PSU's AI tools least. What does that match or miss in your experience?
If the chatbot leaked data, who would be responsible?
When you have to use a tool, does trust still matter?

Ethics and IRB

IRB clearance for the survey and the guide, with an amendment for the guide before interviews.
Informed consent; link survey and interview data by an anonymous code only.
No identifiers beyond what you need; vulnerability details go to the CISO only.
Data encrypted and retained as the IRB requires; GenAI use logged.

Do

Write survey items with one idea each, and adapt validated scales.
Build the interview guide after the survey, with probes tied to results.
Get IRB clearance for both instruments, with an amendment for the guide.

Don't

Ask double-barrelled or leading items.
Ask the same questions in the survey and the interview.
Collect identifiers you do not need.

Mistakes students make here

Student writes: Asking the same questions in the survey and the interview.
Why it fails: The interview then repeats the survey instead of explaining it.
Fix: Use the interview for why and how, tied to survey results.

Student writes: Writing the interview guide before the survey is analysed.
Why it fails: In an explanatory design the guide should respond to the results.
Fix: Draft the guide after Phase 1, and file an IRB amendment.

Student writes: "Don't you think the vendor should be blamed?"
Why it fails: Leading probe. It suggests the answer.
Fix: "Who is responsible if the chatbot leaks data, and why?"

SMART check for this step

S: SpecificOne instrument per strand, each mapped to the RQs.
M: MeasurablePilot alpha of 0.70 or above; interview guide piloted with 2 people.
A: AchievableSurvey takes 12 minutes or less.
R: RealisticInterview guide has 10–12 open questions.
T: Time-boundIRB cleared before the survey; amendment before interviews.

In-class check

Give students one leading probe and one double-barrelled survey item ("I am concerned about the privacy and security of my data"). They rewrite both in two minutes.

Chapter 3: Research design and methods · Step 17 of 23 · about 6 min

3.5Data collection plan

Teach this

Fieldwork in a sequential design is a chain. The interviews cannot start until the survey is closed and analysed, so any delay moves everything after it.

Worked example: PSU

When	Activity	Objective
Weeks 1–6	Survey, with reminders on day 7 and 14; track responses by group weekly	RO1
Weeks 7–8	Analyse survey; select interviewees	RO1, RO2
Weeks 9–14	Interviews with consent; record and transcribe	RO2
Weeks 15–18	Code, integrate, build the joint display	RO3

Do

Keep a fieldwork log: dates, response rates, problems, decisions.
Protect anonymity and store data encrypted, with backups.
Get an IRB amendment before changing any instrument.

Don't

Change instruments midway without approval.
Pressure people to respond.
Publish vulnerability details that could help an attacker.

Mistakes students make here

Student writes: Starting interviews while the survey is still open.
Why it fails: The selection rule cannot be applied, and the design becomes concurrent.
Fix: Close the survey, analyse, then select.

Student writes: No reminders and no weekly response tracking.
Why it fails: The response rate drops, and small groups fall below their minimum.
Fix: Send reminders on day 7 and 14 and track responses by group.

Student writes: Recording interviews before the IRB amendment is approved.
Why it fails: The data cannot be used, and it may be an integrity breach.
Fix: No recording until the amendment is cleared.

SMART check for this step

S: SpecificInstrument, channel and reminders are defined.
M: MeasurableProgress tracked weekly against target.
A: AchievableGatekeeper support for distribution is confirmed.
R: RealisticA window of 4–6 weeks per phase.
T: Time-boundClosing date set before you begin.

In-class check

Students sketch their timeline as a chain. Ask: if the survey closes two weeks late, which dates move, and which milestone is at risk?

Chapter 3: Research design and methods · Step 18 of 23 · about 12 min

3.6Statistical methods and evaluation metrics

Teach this

Analyse each strand completely before integrating. In the survey, check assumptions and report effect sizes. In the interviews, use a codebook and a trustworthiness plan. Then classify every meta-inference.

Worked example: PSU

Item	Method	Decision rule
Screening and reliability	Missing values, attention check, Cronbach's alpha	Missing below 5%; alpha of 0.70 or above
H4 groups	Welch ANOVA with Games-Howell post hoc; Kruskal–Wallis if assumptions fail	p below .05; report η² (0.01 small, 0.06 medium, 0.14 large)
H1, H2, H3	Multiple regression with controls (group, age band, AI use frequency); check VIF below 5 and residuals	β sign as hypothesised; 95% CI excludes 0; Holm correction across the four tests
Interviews	Reflexive thematic analysis (Braun and Clarke, 2006) with a codebook	Second coder on 20% of transcripts; kappa of 0.70 or above
Trustworthiness	Member checking with 4 participants; audit trail (Lincoln and Guba, 1985)	Themes confirmed or revised
Integration	Joint display; classify each meta-inference (Fetters et al., 2013)	Convergence, expansion or discordance stated in every row

Do

Analyse each strand fully before you integrate.
Check assumptions before ANOVA and regression; use Welch or Kruskal–Wallis if they fail.
Code interviews with a codebook and audit trail, then classify each meta-inference as convergence, expansion or discordance.

Don't

Run many tests until one is significant.
Report only the interview quotes that fit.
Infer causation from cross-sectional data.

Mistakes students make here

Student writes: ANOVA without checking equal variances.
Why it fails: Unequal group sizes and variances can invalidate the result.
Fix: Use Welch's ANOVA with Games-Howell, or Kruskal–Wallis if assumptions fail.

Student writes: Interview coding with a single coder, no codebook and no audit trail.
Why it fails: Nobody can check the themes.
Fix: Keep a codebook, use a second coder on 20% and record decisions.

Student writes: Presenting quotes as results with no meta-inference.
Why it fails: Quotes decorate the numbers but nothing is integrated.
Fix: State the meta-inference in each row: convergence, expansion or discordance.

SMART check for this step

S: SpecificEach hypothesis has a named test and threshold.
M: MeasurableAlpha 0.70, p below .05 (Holm), η², β with CI; kappa 0.70 if two coders.
A: AchievableSPSS or R and NVivo skills confirmed.
R: RealisticEach strand analysed within 4 weeks.
T: Time-boundIntegration finished within 2 weeks after the last interview.

In-class check

Exercise: the standard ANOVA gives F(3, 408) = 11.9. Compute η². Answer: η² = (3 × 11.9) / (3 × 11.9 + 408) = 35.7 / 443.7 = 0.08, a medium effect.

Ask: how would you describe an η² of 0.08 to a non-statistician?

Chapter 4: Results and findings · Step 19 of 23 · about 10 min

4.1Results and findings (illustrative)

Teach this

Report each strand alone first, then present the joint display. Discordance is a finding. When the strands disagree, the student learns something neither strand could show.

Worked example: PSU

Illustrative results. Replace with your own. Survey: 412 valid responses (71% of 580 invited): students 208, faculty 82, administrators 61, IT staff 61. Alpha: risk .88, trust .90, governance perception .84, continued use .86.

Hypothesis	Result	Decision
H4 groups differ	Mean perceived risk: students 3.4, faculty 3.7, administrators 3.9, IT staff 4.2. Welch F(3, 168) = 12.4, p below .001; η² = .08 (standard ANOVA F(3, 408) = 11.9). IT vs students p below .001; administrators vs students p = .03	Supported
H1 risk lowers trust	β = −0.46, 95% CI [−0.55, −0.37], p below .001; model R² = .24	Supported
H2 trust raises use	β = 0.09, 95% CI [−0.02, 0.20], p = .11	Not supported
H3 governance lowers risk	β = −0.31, 95% CI [−0.40, −0.22], p below .001	Supported

Interviews: 13 participants (students 3, faculty 3, administrators 4, IT staff 3). Themes: T1 nobody owns the vendor tool (11 of 13); T2 we do not know where the data goes (10 of 13); T3 rules exist but are not enforced (8 of 13); T4 we have to use it, so trust is beside the point (9 of 13).

Joint display

Survey finding	Interview theme	Meta-inference	Meaning
IT staff and administrators report the highest risk and lowest trust	T1, T2	Expansion	Closeness to vendor arrangements exposes accountability gaps (supports P1, P2)
Trust does not predict continued use (H2 not supported)	T4	Expansion (explains a null)	Use is mandated, so persuading users is not the lever
Governance perception lowers risk (H3)	T3	Confirmation with nuance	Rules help only when visibly enforced
Students report the lowest risk	T1: 3 of 3 students could not say who owns chatbot data	Discordance	Low concern may reflect low awareness, not safety

Do

Report each strand separately first, then the joint display.
Show where the strands disagree and interpret the discordance.
Tie each meta-inference back to a proposition.

Don't

Present quotes as if they were statistics.
Hide discordant findings.
Mix recommendations into the results.

Mistakes students make here

Student writes: "Most people said trust was low."
Why it fails: Counting words with no numbers. It mixes the two strands' logic.
Fix: Use numbers from the survey and themes from the interviews.

Student writes: Two separate results sections and no joint display.
Why it fails: Integration is missing. The reader must do it.
Fix: Build the display: finding, theme, meta-inference, meaning.

Student writes: Leaving out the students' discordance.
Why it fails: It looks like cherry-picking, and it is the most interesting finding.
Fix: Report it and interpret it: low concern may mean low awareness.

SMART check for this step

S: SpecificOne table per strand and one joint display.
M: MeasurableEvery row cites a statistic and a theme.
A: AchievableOnly data from approved instruments.
R: RealisticThe chapter fits the word budget.
T: Time-boundDrafted within 6 weeks of the last interview.

In-class check

Give students one survey result (trust does not predict use, β = 0.09, p = .11) and one interview theme ("we have to use it"). They write the joint display row and classify the meta-inference. Answer: expansion, explaining a null result.

Chapter 5: Conclusions, recommendations and further research · Step 20 of 23 · about 6 min

5.1Conclusions

Teach this

Conclusions must draw on both strands where they apply and must say what integration added. Cross-sectional data support "is associated with", never "causes".

Worked example: PSU

Concern about AI security at PSU is real and uneven: IT staff and administrators perceive the most risk and administrators trust least. Risk lowers trust, and visible governance lowers risk. Trust does not drive continued use because use is mandated, so persuasion is not the lever. Interviews trace the pattern to unclear ownership of vendor-run tools (P1) and opaque data flows (P2). The discordant result for students suggests low awareness rather than low risk.

Not shown: whether views change over time, or whether they hold at other universities.

Do

Link each conclusion to an objective, a hypothesis or proposition, and the literature.
State what the evidence supports and what it does not.
Say what changed in your understanding.

Don't

Introduce new data or new literature.
Claim broad generalisation from one case.
Repeat results without interpreting them.

Mistakes students make here

Student writes: Concluding only from the survey.
Why it fails: The interviews then look unnecessary.
Fix: Cite both strands wherever both bear on the point.

Student writes: "Low trust causes administrators to avoid AI tools."
Why it fails: Causal language from cross-sectional data.
Fix: Say "is associated with".

Student writes: Ignoring the discordant result.
Why it fails: Discordance is part of the answer to RQ3.
Fix: Mention it and state what would resolve it.

SMART check for this step

S: SpecificOne conclusion per objective.
M: MeasurableEach cites the statistic or finding it rests on.
A: AchievableWithin what the data support.
R: RealisticScaled to a single case.
T: Time-boundDrafted within 4 weeks of finishing Chapter 4.

In-class check

For each conclusion the student must name the strand or strands it rests on. A conclusion with no source is removed.

Chapter 5: Conclusions, recommendations and further research · Step 21 of 23 · about 10 min

5.2SMART recommendations

Teach this

Recommendations rest on the integrated findings. Each must pass all five SMART tests, and the R is Realistic in your guide.

Worked example: PSU

Chain shown in each row: Problem → Proposition → Conclusion → Recommendation. The conclusions are conditional on your real results.

Chain	Recommendation	Specific	Measurable	Achievable	Realistic	Time-bound
SP1 unknown extent → H4 → administrators and students differ	R1 Train the groups with the largest gaps	Role-based AI security training for administrators, with a short module for students	90% or more completion; at least 20% post-test gain	Delivered through the LMS	Half-day module	Within 6 months
SP2 unknown reasons → P1 → nobody owns vendor tools	R2 Add accountability clauses to AI vendor contracts	Procurement adds standard clauses, including breach notice	100% of new and 80% of renewed contracts	Legal template exists	Applied at renewal cycles	Within 12 months
SP3 no action priority → joint display → incidents unreported	R3 Open an AI incident reporting channel	Online form routed to IT security	At least 80% of staff aware in a pulse survey	Uses the existing helpdesk	No new tools needed	Within 3 months

Do

Keep the chain: Problem → Hypothetical solution or proposition → Conclusion → Recommendation.
Check each recommendation on S, M, A, R and T (R means Realistic in your guide).
Name an owner and a metric.

Don't

Recommend anything your data does not support.
Write generic advice ("improve awareness").
Ignore budget and staff capacity.

Mistakes students make here

Student writes: "Train everyone on AI security."
Why it fails: No target group, metric or deadline.
Fix: Train the group with the lowest trust, with a completion and post-test target.

Student writes: A recommendation supported by only one strand.
Why it fails: It ignores the design's own logic.
Fix: Cite the joint display row it comes from.

Student writes: No thought to capacity.
Why it fails: PSU ignores recommendations it cannot deliver.
Fix: Use existing tools and roles, and say so.

SMART check for this step

S: SpecificOne action, one owner.
M: MeasurableA metric and a target.
A: AchievableWithin the owner's authority.
R: RealisticWithin budget and staff capacity.
T: Time-boundA deadline counted from adoption.

In-class check

Five-minute rewrite: "Train everyone on AI security" becomes a SMART recommendation. Compare versions and vote on the weakest of S, M, A, R and T.

Chapter 5: Conclusions, recommendations and further research · Step 22 of 23 · about 4 min

5.3Areas for further research

Teach this

Further research comes from limitations and from the discordant results. Each item names a design that could resolve it.

Worked example: PSU

Limitation	Further research	Design	Horizon
Cross-sectional	Test whether views and the explanations change	Repeat the survey after 12 months; re-interview a subsample	12–18 months
One institution	Compare universities of different size and vendor mix	Multi-site sequential mixed methods	18–24 months
Trust did not predict use	Test what does drive continued use under mandate	Add habit and mandate constructs; consider PLS-SEM	12 months
Students' low risk may be low awareness	Test an awareness intervention	Quasi-experiment with pre and post measures	12 months

Do

Base each item on one of your own limitations or unanswered findings.
Say which design would answer it.
Keep to 3–5 items.

Don't

Write "more research is needed" with no direction.
List topics unrelated to your findings.
Promise to do it all yourself.

Mistakes students make here

Student writes: "Do a bigger survey."
Why it fails: Not linked to a limitation or a finding.
Fix: Say what a bigger survey would resolve.

Student writes: Ignoring the discordant finding.
Why it fails: It is the most natural next question.
Fix: Test whether low concern reflects low awareness.

Student writes: Listing eight ideas.
Why it fails: Length hides priority.
Fix: Keep three to five.

SMART check for this step

S: SpecificEach item names a limitation and a design.
M: MeasurableEach states what result would count as an answer.
A: AchievableFeasible for a future research team.
R: RealisticWithin reach of one funded project.
T: Time-boundSuggests a horizon, for example 12–24 months.

In-class check

Each student turns one limitation into a further-research item with a design and a horizon.

Before you submit · Step 23 of 23 · about 8 min

6Traceability and submission check

Teach this

Traceability in a mixed study has an extra row: the integration. Every recommendation must trace to a joint display row, not to one strand alone.

Worked example: PSU

Sub-problem	Objective	Question	Hypothesis or proposition	Data	Test	Recommendation
SP1	RO1	RQ1	H1–H4	Survey, 400 valid	Regression, Welch ANOVA	R1 targeted training
SP2	RO2	RQ2	P1, P2	12–15 interviews	Thematic analysis	R2 vendor clauses
SP3	RO3	RQ3	Meta-inferences	Both strands	Joint display	R3 incident channel

If any cell is empty, the design is broken. Fix it before you write more.

Do

Check that every sub-problem has an objective, question, hypothesis or proposition, data, test and recommendation.
Run the check before every milestone.
Fix empty cells before you write more.

Don't

Add a recommendation with no finding behind it.
Leave a hypothesis with no test.
Change one link without updating the others.

Mistakes students make here

Student writes: SP3 has no row for the meta-inferences.
Why it fails: The integration has no home, so the study ends without a decision.
Fix: Add the integration row with its data and test.

Student writes: Interviews with no proposition.
Why it fails: There is nothing for the interviews to test or explain.
Fix: Link each theme to P1 or P2.

Student writes: A recommendation with no joint display row behind it.
Why it fails: It ignores the design's logic.
Fix: Trace each one back to the display.

SMART check for this step

S: SpecificEvery row is complete.
M: MeasurableZero empty cells.
A: AchievableBuilt from your own chapters, not from this page.
R: RealisticTakes under an hour.
T: Time-boundDone before each milestone.

In-class check

Peer audit: students swap traceability tables and mark every empty cell in red. Ten minutes. Each student then fixes the cells their partner found.

Anchor references

Starting points named in this walkthrough. Open each one and confirm the details before you cite it or teach from it.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs: Principles and practices. Health Services Research, 48(6, Pt 2), 2134–2156.
Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? Field Methods, 18(1), 59–82.
Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3), 607–610.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Sage.
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734.
National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0).
Norberg, P. A., Horne, D. R., & Horne, D. A. (2007). The privacy paradox: Personal information disclosure intentions versus behaviors. Journal of Consumer Affairs, 41(1), 100–126.
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478.

Objective (SP, RQ)	Specific	Measurable	Achievable	Realistic	Time-bound
RO1 Validate measurement (SP1, RQ1)	Reflective measurement model for all five constructs	Loadings 0.708 or above; CR 0.70–0.95; AVE 0.50 or above; HTMT below 0.85	Adapted validated scales	Pilot with 30 respondents first	t0 + 4 months
RO2 Test structure (SP2, RQ2)	Paths H1–H5 and mediation H3	Bootstrap 5,000; 95% CIs; report R², f², Q²	Data already in the survey	Standard SmartPLS workflow	t0 + 5 months
RO3 Compare groups (SP3, RQ3)	Four stakeholder groups (H6)	MICOM passed; multi-group analysis on the risk-to-trust path; at least 60 per group	Oversample small groups	Limited to one key path	t0 + 6 months

Hypothesis	Path and sign	Theory reason	Decision rule
H1	Perceived risk → Trust (negative)	Trust theory (Mayer et al., 1995)	95% CI excludes 0; f² reported
H2	Trust → Continued use (positive)	Acceptance models extended with trust (Venkatesh et al., 2003)	95% CI excludes 0
H3	Perceived risk → Trust → Continued use (indirect, negative)	Trust as mediator	Indirect effect 95% CI excludes 0
H4	Governance maturity → Perceived risk (negative)	Assurance mechanisms lower perceived risk (NIST AI RMF logic)	95% CI excludes 0
H5	Awareness → Perceived risk (negative). The sign is contested.	Awareness reduces uncertainty; rival view: awareness makes threats more salient	95% CI excludes 0 and sign as stated
H6	Stakeholder group moderates Perceived risk → Trust	Role and expertise change how risk is weighed	MICOM passed; permutation p below .05 after Holm correction

Construct	Role	Type	Indicators	Basis
Governance maturity (GM)	Exogenous	Reflective	4	NIST AI RMF Govern function, as seen by users; new items
Security awareness (AW)	Exogenous	Reflective	4	New items; expert review
Perceived risk (PR)	Mediator	Reflective	5	Adapted risk and privacy-concern scales
Trust (TR)	Mediator	Reflective	5	Adapted from McKnight et al. (2002)
Continued use (CU)	Endogenous	Reflective	4	Adapted from Bhattacherjee (2001)

Limitation	Mitigation
Cross-sectional data: no causal claims	Say "predicts" and "is associated with"; name a panel study as further research
Common method bias (one survey, one source)	Anonymity, varied item wording, and a full collinearity VIF below 3.3 (Kock, 2015)
Non-response bias	Compare early and late responders (Armstrong and Overton, 1977)
Single institution	Describe PSU in enough detail for readers to judge transfer

Hypothesis (RQ)	Supports	Challenges	Neutral	Critical evaluation
H1 risk → trust (RQ2)	Perceived risk lowers trust (Mayer et al., 1995)	Privacy paradox (Norberg et al., 2007); mandated use	Effect depends on context	Mostly e-commerce, not university AI
H2 trust → use (RQ2)	Trust raises intention in acceptance models (Venkatesh et al., 2003)	Habit and mandate override intentions	Effect depends on voluntariness	Intention is not behaviour
H3 mediation (RQ2)	Trust mediates risk effects on use	Direct effects may dominate	Depends on voluntariness	Cross-sectional mediation cannot show order in time
H4 governance → risk (RQ2)	Assurance and governance lower perceived risk	Policies are invisible to users; compliance can be symbolic	Works only when visible	Few studies measure governance from the user's view
H5 awareness → risk (RQ2)	Awareness reduces uncertainty and calibrates risk	Awareness makes threats more salient and raises perceived risk	Depends on training content	The direction is contested; treat it as an open question
H6 group differences (RQ3)	Role and expertise change how risk is weighed	Culture matters more than role	Differences are small	Group comparison needs invariance testing, which is rarely done

Research question	Objective	Sub-problem	Answered by
RQ1 Can governance maturity, awareness, perceived risk, trust and continued use be measured reliably and validly at PSU?	RO1	SP1	Measurement model
RQ2 To what extent do governance maturity and awareness predict perceived risk, perceived risk predict trust, and trust predict continued use, including mediation?	RO2	SP2	Structural model, bootstrap
RQ3 Do these effects differ by stakeholder group?	RO3	SP3	MICOM and multi-group analysis

Step	Task	Output	Deadline
1 Pilot	30 respondents; check loadings and wording	Revised instrument	t0 + 2 months
2 Survey	Stratified sample of about 580 invited; 6-week window	400 valid responses	t0 + 4 months
3 Measurement model	Loadings, CR, AVE, HTMT, full collinearity VIF	Validated constructs (RQ1)	t0 + 4 months
4 Structural model	Bootstrap 5,000; R², f², Q², PLSpredict; mediation	Paths H1–H5 (RQ2)	t0 + 5 months
5 Groups	MICOM, then multi-group analysis	H6 result (RQ3)	t0 + 6 months

Alternative considered	When it would be better, and why not here
CB-SEM	Better for confirming a fixed theory with a global fit test. Here the theory is still developing and prediction matters. Run CB-SEM as a robustness check if your committee asks for global fit
Multiple regression	Ignores measurement error and cannot test the whole model with mediation at once
Interviews or case study	Cannot test hypothesised relationships or compare groups statistically

Item	Decision	Reason
Population and frame	About 10,000: 8,000 students, 1,200 faculty, 600 administrators, 200 IT staff; PSU registry	A complete list allows random sampling within groups
Sampling	Stratified random with group minimums. Target: students 208, faculty 82, administrators 61, IT staff 61	Multi-group analysis needs at least 60 per group
Inverse square root method	About 155 needed to detect a path of 0.20 at 80% power, 5% significance (Kock and Hadaya, 2018)	Rule-based, not a 10-times rule
G*Power check	About 68 needed for f² = 0.15, power .80, 2 predictors (Faul et al., 2009)	Confirms the model is well powered
Final target	400 valid (at least 380 by Krejcie and Morgan, 1970). About 580 invited at 70% response	Covers both the model and the group analysis
Weights	Groups are over-represented relative to their share, so report group-level results and include group as a control in pooled models	Avoids biasing pooled estimates

Construct	Items	Sample item (7-point agree scale)	Source
Governance maturity (GM)	4	PSU has clear rules about who is responsible for each AI tool.	New items; 3 experts check content validity
Security awareness (AW)	4	I know how to protect my data when I use PSU's AI tools.	New items; expert review
Perceived risk (PR)	5	I am concerned that my personal data could be exposed when I use PSU's AI tools.	Adapted risk and privacy-concern scales
Trust (TR)	5	I believe PSU's AI tools handle my data responsibly.	Adapted from McKnight et al. (2002)
Continued use (CU)	4	I intend to keep using PSU's AI tools next semester.	Adapted from Bhattacherjee (2001)
Controls and checks	4	Group, age band, AI use frequency, one attention check	Standard

When	Activity	Objective
Weeks 1–2	Expert content check of new items; pilot with 30	RO1
Weeks 3–8	Main survey, with reminders on day 7 and 14; track responses by group weekly and oversample any group below 60	RO1
Week 9	Screen data; early versus late responder comparison	RO1
Weeks 10–16	Measurement model, structural model, mediation, groups	RO1–RO3

Step	What you check	Threshold or rule
0 Screening	Missing values, straight-lining, attention check, outliers; full collinearity VIF for common method bias	Missing below 5% per item; VIF below 3.3 (Kock, 2015)
1 Measurement (reflective)	Outer loadings, composite reliability, AVE, discriminant validity	Loadings 0.708 or above; CR 0.70–0.95; AVE 0.50 or above; HTMT below 0.85 (Henseler et al., 2015)
2 Structure	Collinearity, paths, R², f², Q², PLSpredict, SRMR (indicative)	Inner VIF below 3; bootstrap 5,000, BCa 95% CI; f² of 0.02, 0.15, 0.35 = small, medium, large
3 Mediation (H3)	Indirect effect through trust	Bootstrap CI excludes 0; classify per Zhao et al. (2010)
4 Groups (H6)	MICOM, then multi-group analysis with permutation	Configural and compositional invariance; Holm-adjusted p below .05 (Henseler et al., 2016)
5 Robustness	Add controls; compare with regression; check non-response	Signs and significance stable

Construct	Items kept	Loadings	α	CR (ρc)	AVE	Max HTMT
GM	4	0.78–0.88	.84	.89	.68	0.71
AW	4 of 5	0.74–0.85	.81	.87	.62	0.71
PR	5	0.76–0.87	.88	.91	.67	0.58
TR	5	0.80–0.90	.90	.93	.73	0.58
CU	4	0.79–0.88	.86	.90	.70	0.44

Hypothesis	Path	β	95% BCa CI	f²	Decision
H4	GM → PR	−0.35	[−0.44, −0.25]	0.13	Supported
H5	AW → PR	+0.12	[0.02, 0.22]	0.02	Not supported: significant, but the sign is opposite
H1	PR → TR	−0.46	[−0.55, −0.36]	0.27	Supported
H2	TR → CU	0.09	[−0.02, 0.20]	0.01	Not supported
H3	PR → TR → CU	−0.04	[−0.09, 0.01]	n/a	Not supported: CI includes 0

Group	PR → TR (β)	Difference from students	Permutation p (Holm-adjusted)
Students	−0.38	n/a	n/a
Faculty	−0.49	−0.11	.21 (.42)
Administrators	−0.62	−0.24	.004 (.024)
IT staff	−0.41	−0.03	.77 (.77)

Chain	Recommendation	Specific	Measurable	Achievable	Realistic	Time-bound
SP2 untested links → H4 supported → governance lowers perceived risk	R1 Adopt a tailored AI security policy	Policy aligned to NIST AI RMF, ISO 27001 and DPDP	Governance score up 30%; perceived-risk mean down 0.5 SD in a pulse survey	Builds on the current ISMS	Fits existing staffing	Policy in 6 months; pulse in 12
SP1 unmeasured constructs → H1 supported → risk lowers trust	R2 Publish an AI system register and data-use notice	Public register with data uses per system	100% of production systems listed; trust mean up 0.3	Uses the audit inventory	One page per system	Within 4 months
SP3 group differences → H5 and H6 → awareness raises risk; administrators most sensitive	R3 Pair awareness training with visible controls, starting with administrators	Every module shows which controls protect users	90% or more completion; perceived-risk mean rises by no more than 0.1 after training	Uses the LMS	Half-day module	Within 6 months

Limitation	Further research	Design	Horizon
Cross-sectional	Test whether the paths hold over time	Two-wave panel, 6 months apart	12–18 months
One institution	Test the model at other universities	Multi-site PLS-SEM or CB-SEM confirmation	18–24 months
Trust did not predict use	Add mandate and habit constructs	Extended model with a voluntariness moderator	12 months
Awareness raised perceived risk	Test which training content calibrates risk	Randomised comparison of two training designs	12 months

Sub-problem	Objective	Question	Hypothesis	Data	Test	Recommendation
SP1	RO1	RQ1	Measurement of 5 constructs	Survey, 400 valid	Loadings, CR, AVE, HTMT	R2 register and notice
SP2	RO2	RQ2	H1–H5	Same survey	Bootstrap 5,000; indirect effect CI	R1 tailored policy
SP3	RO3	RQ3	H6	Same survey; at least 60 per group	MICOM, then multi-group analysis	R3 awareness with visible controls

Teaching the DSR dissertation, step by step, with the PSU case

1.1Background

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.2Problem statement

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.3Sub-problems

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.4Purpose

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.5Research objectives (SMART)

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.6Research questions

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.7Design objectives and design propositions

Design objectives (from RO1)

Design propositions (tested in RO3)

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.8Conceptual framework

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.9Scope, assumptions and limitations

In scope

Out of scope

Assumptions

Limitations and mitigations

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

1.10Contribution to praxis

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

2.1Literature themes mapped to research questions

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

2.2Research and practitioner gap

Do

Don't

Mistakes students make here

SMART check for this step

In-class check

3.1Research design