Onboarding Research Refresh
Onboarding Research Refresh
Prompt-first procedure: Describe the outcome you want in your agent conversation. The agent should select and load the appropriate AIWG assets, explain material changes, request any needed approval, and report verification evidence. Exact commands and flags appear only in the CLI reference.
First time using AIWG? Begin with Install, Connect, and Verify. This guide assumes AIWG is already installed, `all` is deployed for your provider, and the agent-owned context regeneration procedure has connected the agent to this project.
This note refreshes the evidence base for beginner onboarding. The practical problem is narrow: AIWG needs first-run guidance that helps a beginner build the right mental model, calibrate trust, and verify what the agent actually did.
We chose a bounded refresh instead of a broad literature review because the docs decision is already scoped. While 5 sources are not a full survey, they are enough to test whether the current Start Here pattern is still defensible in 2026.
Validation baseline: revisit this note when AIWG version 2026.5 onboarding claims change. While the source set is small, maintainers should record any issue that failed because a newer study contradicts the current trust-calibration guidance.
Accepted Sources
| Source | Why it matters for AIWG |
|---|---|
| IBM Research / IUI 2025, Building Appropriate Mental Models: What Users Know and Want to Know about an Agentic AI Chatbot | Directly studies an agentic chatbot. N=24 participants often understood the system as search-like but missed the generative model and agentic framework behind actions. |
| Google Research / CHI EA 2024, Experiencing InstructPipe | Studies novice construction of AI pipelines. The formative work used 58 contributors and 236 proposed pipelines, which supports AIWG's plain-language-to-workflow translation path. |
| Scharowski et al. / FAccT 2025, To Trust or Distrust AI: A Questionnaire Validation Study | Validates trust/distrust measurement in an AI setting with N=1485 and argues that trust and distrust should be measured separately. This matters for onboarding surveys and dry-run notes. |
| IJHCS 2025, Appropriate reliance in XAI systems | Clarifies trust, distrust, plus reliance. Useful background for avoiding "increase trust" language; the goal is appropriate reliance. |
| Steinmetz et al. 2025, The Trust Calibration Maturity Model | Proposes five trustworthiness communication dimensions covering performance, bias/robustness, transparency, safety/security, plus usability. Useful as a checklist for what the status probe should and should not claim. |
Synthesis
We found one repeated pattern across these sources: users need visible system-action transparency, not more branding. AIWG should show the project root, deployed provider files, installed frameworks, and next command because those are observable. It should avoid claiming that a provider session will behave correctly unless there is field evidence for that provider.
The onboarding implication is concrete:
1. Keep Start Here focused on one path. 2. Translate ordinary user goals into 2 to 4 the agent’s capability search phrases. 3. Require the agent’s stable-asset loader before recommending a capability. 4. Use the agent-owned status operation as a local evidence surface. 5. In validation notes, record trust and distrust signals separately.
Design Decisions Reinforced
| AIWG decision | Evidence support |
|---|---|
| Status probe reports observable facts, not passive attribution. | IBM/IUI 2025 and TCMM both support transparency about actions and limits. |
| Beginner docs avoid framework catalogs. | InstructPipe reinforces text-to-workflow translation for novices. |
| Provider behavior remains gated on field evidence. | Trust-calibration sources warn against overclaiming system capability. |
| Validation uses dry-run notes, not mandatory telemetry. | FAccT 2025 and IJHCS 2025 support careful measurement of reliance, trust, plus distrust. |
Induction Follow-Up
These sources should be considered for formal induction in the shared research corpus if this evidence becomes release-critical. The strongest candidates are the IBM/IUI 2025 agentic mental-model paper, the FAccT 2025 trust/distrust validation paper, plus the IJHCS 2025 trust/reliance survey.
Use the same 80% evidence discipline from the provider matrix when turning this note into release claims: a source can support a design decision, but it does not prove provider behavior.