Introduction to Questionnaire of Self-Efficacy and Needs in Using Large-Language Model-Based AI Services (Q-SNELL)
Introducing Q-SNELL: A Tool for Evaluating Self-Efficacy and Needs in Using Large Language Model AIs
The rapid adoption of Large-Language Model-based AI Services (LLMAIs), like ChatGPT and Claude, have highlighted the need to understand users' self-efficacy and specific needs when engaging with these technologies. Such understanding is crucial for evaluating the effectiveness of LLMAI applications and guiding their future development. However, existing assessment tools often fall short in capturing these dimensions.
To address this gap, we developed and validated the Questionnaire of Self-efficacy and Needs in using core features of LLMAIs (Q-SNELL). This tool was meticulously designed based on a comprehensive review of LLMAI core features and validated through feedback from experts and users. The Q-SNELL includes three parts:
- Self-efficacy and needs evaluation across eight core LLMAI features.
1. Text Generation
2. Text Modification
3. Text Analysis, Key Point Extraction, or Text Synthesis
4. Seeking Information, Recommendations, or Knowledge
5. Setting Plan Objectives, Steps, Timelines, and Considerations
6. Language Learning or Translation
7. Writing, Modifying, or Learning Code
8. Casual Chatting - Feature prioritization to identify the most frequently used functions.
- General self-efficacy evaluation.
The Q-SNELL stands out for its exceptional flexibility, making it a versatile tool for researchers with diverse objectives. Its design allows for the selective application of items, enabling researchers to focus on specific aspects of the LLMAI user experience that align with their study goals.
Key Features of Q-SNELL
- Assesses self-efficacy and needs across eight core LLMAI features as well as general self-efficacy in using LLMAIs.
- Provides an index of training outcomes, offering insights to guide the development of training programs for effective LLMAI usage.
- Offers a flexible design, allowing users to selectively choose items tailored to specific research or educational objectives.
Access Q-SNELL
Download the Q-SNELL questionnaire via the link below:
For any questions or feedback, feel free to contact us at juyujeng@gmail.com
We hope Q-SNELL will support your research or teaching efforts in understanding and optimizing interactions with LLMAIs!
Q-SNELL：評估使用大型語言模型AI服務之自我效能與需求
隨著基於大型語言模型建立的 AI 服務（LLMAIs）迅速發展（例如 ChatGPT 和 Claude），理解使用者在使用這些人工智慧服務時的自我效能與需求越發重要。這些理解對於評估LLMAI應用的有效性以及啟發這些服務未來發展方向至關重要。然而，現有的評估工具往往無法充分捕捉這些關鍵面向。
為了解決這一不足，我們開發並驗證了 大型語言模型 AI 服務核心功能使用的自我效能與需求問卷（Q-SNELL）。此工具對LLMAI核心功能的做了完整的回顧，並透過專家與使用者的回饋進行題目設計與驗證。Q-SNELL 包括三個部分：
- 八大核心功能的自我效能與需求評估：
1. 產生文本
2. 修改既有文本
3. 分析、摘錄重點、或匯整文本
4. 尋求資訊、建議或知識
5. 制定計畫的目標、步驟、時程與注意事項等
6. 學習語言或翻譯
7. 撰寫、修改或學習程式碼
8. 休閒聊天 - 最常使用的核心功能，用以識別最常使用的功能。
- 整體自我效能評估。
Q-SNELL 的設計極具使用彈性，適用於多元研究目標。其設計允許研究者選擇性地施測與研究目標相關的核心功能有關的問卷項目。
Q-SNELL 的關鍵特點
- 評估使用者在八大核心LLMAI功能之使用自我效能和使用需求，以及對LLMAI的使用整體自我效能。
- 提供訓練成果指標，為LLMAIs操作能力訓練提供學習成效指標。
- 提供靈活的設計，允許使用者根據特定研究或教育目標選擇特定核心功能相關的問卷項目。
下載 Q-SNELL
您可以透過以下連結下載 Q-SNELL 問卷：
如有任何問題或回饋，請隨時聯絡我們：juyujeng@gmail.com
我們希望 Q-SNELL 能協助您在研究或教學中更好地理解與最佳化使用LLMAI的經驗！