Generative Materials Evaluation
This domain tests AI models' ability to generate comprehensive, compliant safety training materials and documentation that meet regulatory standards.
Test Scenario
Domain:
Training
Assigned To:
Jason Allen
Title:
Forklift Training for New Operators
Test Prompt
You are an EHS Professional responsible for developing a Powered Industrial Truck (PIT) training program for new forklift operators.
Your company was cited under OSHA 1910.178(l) after a new hire was found operating a forklift without completing all the required training elements as required by OSHA. The operator had only viewed a brief PowerPoint presentation and was allowed to drive without completing hands-on instruction or a documented performance evaluation.
Using OSHA 1910.178 and the principles of ANSI/ASSP Z490.1, create a training program for new PIT operators that includes the following:
- Formal instruction
- Practical (hands-on) training
- Performance evaluation
Your training program should also follow Z490.1 elements, including a needs assessment, learning objectives, training delivery strategy, evaluation methodology, documentation plan, and a process for continuous improvement.
Include a simple one-page checklist that a qualified evaluator can use to verify training completion before an operator is cleared to use a forklift independently.
Expected Output
1. Needs Assessment
- Identifies the regulatory failure that triggered the training need
- Explains risk factors associated with untrained PIT operators
- Defines the scope of trucks and environments that training covers
2. Learning Objectives
- 3-5 clear, measurable objectives
- Aligned with both safety outcomes and OSHA expectations
3. Training Design & Delivery Plan
- Describes how formal instruction will be delivered (classroom, video, etc.)
- Matches methods to adult learners (e.g., visual demos, scenario-based coaching)
- Includes structured sequencing
- Training must reflect the specific type(s) of trucks used and conditions in the actual work environment
4. Evaluation Method
- Describes how both knowledge and hands-on performance will be evaluated
- Identifies that the evaluation must be conducted by a qualified person
- Defines pass/fail criteria
- Describes how the evaluation will be documented and retained
5. Documentation Plan
- Lists records to be kept (sign-in sheets, evaluation forms, training dates)
- Describes how to confirm training completion before equipment use
6. Continuous Improvement
- Suggest training review based on incidents, audits, or regulation updates
- Include evaluation at least every 3 years, and retraining when an operator has an accident, is observed operating unsafely, or changes in equipment/work area
7. Operator Competency Evaluation Checklist
- A practical one-page tool for documenting the final performance evaluation
- Used by a qualified person to observe and assess operator behaviors
- Includes space for evaluator name, date, pass/fail status, and signature
Grading Rubric
Each submission is blind scored by certified safety professionals using this comprehensive rubric:
| Component | Pass/Fail Criteria | Usefulness Score (0-10) |
|---|---|---|
| 1. OSHA 1910.178 Compliance | Training must address all required elements under 1910.178(l) | Regulatory accuracy |
| 2. Needs Assessment | Identifies compliance issue and training trigger | Context relevance |
| 3. Learning Objectives | Clear, measurable, and relevant to PIT operation | Instructional quality |
| 4. Training Design & Delivery | Includes all 3 elements: formal, practical, evaluation | Completeness |
| 5. Evaluation Method | Clear method to judge and document competence | Assessment validity |
| 6. Documentation Plan | Includes records, signatures, verification process | Audit readiness |
| 7. Continuous Improvement | Addresses how/when training will be reviewed and updated | Long-term viability |
| 8. Operator Competency Evaluation Checklist | Practical, usable form for real-world sign-off | Field utility |
| Overall Usefulness | Would a real EHS professional use this without major edits? | |
Scoring System:
- Fail: If any of the first 8 items are missing or inadequate
- 9-10: Ready to implement with little to no edits
- 7-8: Useful with minor edits
- 5-6: Some value, but not field-ready
- <5: Not practically usable
Evaluation Process
AI models are provided with the prompt above and must generate complete, compliant training materials. Responses are evaluated by Jason Allen and other certified safety professionals for regulatory compliance, practical utility, and completeness. This tests the model's ability to understand complex safety regulations and generate professional-grade documentation that EHS professionals can use in real-world applications.