Evaluate a Tool

Assess an existing, market-available AI tool for DoD medical-facility delivery. First gate it for DoD feasibility (Tier 1–3), then score it 1–5 across five weighted criteria for a 0–100 result. Work top to bottom, or turn on Guided mode (top-right) for a step-by-step walkthrough — and open any “How to fill this section” bar for help.

Tool Info & Scoring

Tier --
1

Tool basics

  • Tool Name — the specific commercial product & vendor (e.g., "Autodesk Forma"). Evaluate existing tools, not concepts.
  • Lifecycle Phase — where it primarily applies: Planning & Programming, Design (BIM/VDC), Construction, or Operations & Lifecycle.
  • Choose the phase of greatest impact; note any secondary or cross-cutting use in the SME comments.
2

AI Category Classification

Pick what the tool's AI mainly does. Tap ? for what each category means.
  • Pick the Primary Category for the tool's core AI capability — this complements the lifecycle phase (required).
  • Add a Secondary Category only if it genuinely spans two capabilities — otherwise leave "None".
  • "AI" means real ML, computer vision, NLP, generative, or predictive capability — not marketing claims. Hover a dropdown option for examples.
3

Tier Gate (Feasibility)

Answer Yes / Partial / No from evidence — this decides the tier before scoring. Tap ? for what each question means.
  • This is a feasibility gate, applied before scoring so infeasible tools aren't elevated by features alone.
  • Enterprise support (built for regulated orgs) · DoD-compatible deployment (cloud / on-prem / FedRAMP) · basic security (access, encryption, logging) · government precedent.
  • Any No → Tier 3. Any Partial (no No) → Tier 2. Two+ Yes → Tier 1. Base answers on evidence, not impressions.
Enterprise / regulated environment support?
Deployment compatible with DoD?
Basic security controls?
Precedent in government contexts?
Tier 1 if no "No" and 2+ "Yes" | Tier 2 if any "Partial" | Tier 3 if any "No"
4

Weighted Scoring Rubric

Score 1 (poor) to 5 (excellent). Tap ? by any category for exactly what to look for.
  • Score each category 1 (poor) → 5 (excellent) after tiering; half-steps (e.g. 3.5) are allowed.
  • Weights favor the mission — DoD feasibility & UFC relevance over novelty: Feasibility 30% · UFC 25% · Interop 20% · Maturity 15% · ROI 10%.
  • Under 3.0 on Feasibility, UFC, or Interop raises a non-disqualifying soft flag for SME attention.
Weights: Feasibility 30%, UFC 25%, Interop 20%, Maturity 15%, ROI 10%
Category Weight Score Weighted
Soft thresholds: < 3.0 triggers flags for Feasibility, UFC, or Interop.

Evaluation Review Dashboard

0
Total Tools
0
Tier 1 (DoD-Ready)
0
Tier 2 (Possible)
0
Tier 3 (Not Feasible)
Tool Name Lifecycle AI Category ?6 AI capability types: Generative Design, QA/QC, Cost & Schedule, Reality Capture, Digital Twins, and Document Intelligence. Classifies what kind of AI the tool provides. Tier ?Tier 1 (DoD-Ready): Enterprise-grade, near-term feasible for pilot.
Tier 2 (DoD-Possible): Valuable but needs policy/IT changes.
Tier 3 (Not Feasible): Informational only, long-term tracking.
Score ?Weighted 0-100 score across 5 categories:
DoD Feasibility: 30%
UFC Relevance: 25%
Interoperability: 20%
Maturity: 15%
ROI Potential: 10%
80+ High, 70-79 Strong, 60-69 Moderate, <60 Low
SME Status ?Subject Matter Expert validation status. Scores are initially assigned by the research team, then validated by MCX, ERDC, BIM/VDC, or Cybersecurity SMEs at 35%, 65%, and 95% milestones. Date Actions
No evaluations saved yet.

Saved Evaluations Local DB

0 entries
No saved evaluations yet. Complete an evaluation and click "Save to Database".

Reports

Task #5 milestone submittals for the AI Tooling Evaluation Framework (UFC 4-510-01). Open a completed report to read it in full.

100%
Final Submittal
Coming soon

Help & Support

Everything you need to evaluate AI tools consistently and defensibly for DoD medical facilities — the purpose, the tier gate, the weighted scoring, the team workflow, a glossary, and FAQs. Grounded in the Task #5 AI Tooling Evaluation Framework.

Purpose & guiding principles

Task #5 is a structured, defensible way to evaluate existing, market-available AI tools for the planning, design, construction, and lifecycle of DoD medical facilities. The goal is not to build AI — it's to determine what exists today, what is usable within DoD constraints, and how each tool could inform UFC 4-510-01 criteria and workflows.

Five guiding principles

Standards First
AI must support and clarify UFC criteria, not replace them.
Evidence Over Hype
Prefer tools with demonstrated value and documented use.
DoD Feasibility
Deployment model, cybersecurity posture, and data handling are decisive.
Criteria Impact
Always ask how a tool could influence UFC 4-510-01.
Future-Proofing
Support structured, phased adoption — not one-off pilots.
What counts as "AI" here — software using machine learning, computer vision, natural-language processing, generative algorithms, or predictive analytics to automate, augment, or materially improve decisions across the facility lifecycle. Marketing claims without real AI capability don't qualify.

1 How an evaluation works — 5 steps

1
Tool basics

Name the tool and pick the lifecycle phase where it's mainly used (Planning, Design, Construction, or Operations).

2
AI category

Classify what kind of AI it is — primary, plus an optional secondary (e.g., Generative Design, QA/QC, Digital Twins).

3
Tier gate

Answer 4 Yes / Partial / No feasibility questions. These set the Tier: 1 Ready · 2 Possible · 3 Not feasible.

4
Weighted rubric

Score 5 categories 1–5. Weighted into a 0–100 score (Feasibility 30 · UFC 25 · Interop 20 · Maturity 15 · ROI 10).

5
Review & save

Check the Tier, score, flags, and Recommended Path, add SME notes, then Save — it appears in Dashboard & Gallery.

2 Feasibility tiers & weighted scoring

Every tool is tiered for feasibility first, then scored. Tiering is a gate applied before scoring, so infeasible tools can't be elevated by features alone.

Tier 1 — DoD-Ready
Near-term feasible. Enterprise-grade tools with documented use in regulated environments, plausibly operable within DoD cybersecurity and data constraints. (No "No" and two or more "Yes".)
Tier 2 — DoD-Possible
Mid-term feasible. Mature commercial tools that add value but require policy, IT, or contractual changes to deploy. (Any "Partial", no "No".)
Tier 3 — Not Currently Feasible
Long-term / informational. Early-stage or research-driven tools lacking enterprise controls or requiring unrestricted cloud access. (Any "No".)

Weighted rubric — score 1–5, ×weight → /100

DoD / USACE Feasibility — 30%
Deployment model, cybersecurity posture, and data handling within DoD constraints.
UFC & Standards Relevance — 25%
How directly the tool supports or influences UFC 4-510-01 criteria.
Interoperability & Workflow — 20%
Fit with USACE / BIM workflows and ability to exchange shared data.
Maturity & Market Adoption — 15%
Proven deployment, vendor stability, breadth of use.
ROI Potential — 10%
Measurable time / cost / risk benefit relative to effort.

Reading the result

Score bands
80+ High · 70–79 Strong · 60–69 Moderate · under 60 Low.
Soft flags
A category under 3.0 (Feasibility, UFC, or Interop) is flagged — a non-disqualifying risk indicator for SME attention, never an automatic fail.
Recommended Path
Tier 1 + 75+ = Pilot Candidate · Tier 2 = Criteria-Informing · Tier 3 = Informational.
Each evaluation records
Identification + lifecycle, feasibility tier, the weighted scorecard, an SME narrative (strengths, limits, assumptions, medical relevance), and UFC touchpoints.

3 SME validation & milestones

Subject-Matter Experts (MCX, ERDC, BIM/VDC, and Cybersecurity) review tier assignments, soft-threshold flags, and narratives at the 35%, 65%, and 95% milestones. Final judgments are informed by SME review — not scores alone — and every adjustment is documented for transparency and auditability. Use the SME Validation Notes on each evaluation to record who reviewed it, the date, any tier/score changes, and any policy or IT blockers, then set the Validation Status.

4 Team scoring & import

Reviewers score the same tools in the shared Excel workbook; answers combine by weight, then import here.

Brian
×1
weight
Mike
×2
weight
Allison
×2
weight
Van
×3
weight
1
Fill your tab

Open the Team Scoring workbook, go to your colored tab, and score each tool. Guests can use the Guest tab and set their own weight.

2
Combine

The Combined tab weight-averages everyone automatically (anyone who leaves a score blank is skipped).

3
Import

Click Import Team Excel in the toolbar and choose the workbook — it replaces the sample data with your team's weighted results.

§ Glossary

Enterprise-Grade
Software designed for large, regulated organizations.
FedRAMP
Federal authorization program for cloud security compliance.
Cybersecurity Posture
Overall security readiness of a system or vendor.
Data Governance
Policies and controls managing data quality, security, and usage.
PHI
Protected Health Information — sensitive health data requiring protection.
Deployment models
Cloud (vendor-hosted), On-Premises (local secure infra), Hybrid, or Edge.
Interoperability
Ability of systems to exchange and use shared data (e.g., IFC, COBie).
Digital Twin
Dynamic digital representation of a facility linked to performance data.
Generative Design
Algorithm-driven creation of optimized design options.
QA/QC
Quality Assurance / Control — processes ensuring compliance and accuracy (e.g., clash detection).
Computer Vision / NLP
AI that interprets images & video, or analyzes & generates human language.
Predictive Analytics
AI forecasting outcomes from historical data.
UFC Touchpoint
A UFC section or workflow potentially influenced by a tool.
Soft Threshold Flag
A non-disqualifying risk indicator in the framework.
Zero Trust
Security model requiring continuous verification of users and devices.

? Frequently asked

Where is my data stored?

In your team's shared cloud database (Supabase). Everyone signed in reads and writes the same data, and changes sync live between people who have it open. Opened offline as a local file, it falls back to this-device-only storage.

How do I add a new tool?

Click New Evaluation, fill the five sections, and Save. To do many at once, score them in the Team Excel workbook and import it.

Can I export results?

Yes — the Export menu gives a text summary, a single-tool CSV, an all-tools CSV, or a PDF (via print).

What does Reset do?

It clears the current form only. Your saved evaluations are not affected.

Why won't a secondary category select?

The secondary category can't be the same as the primary — choose a different one.

Welcome to the AI Evaluation Matrix

Task #5 assesses existing AI products for the planning, design, construction, and operation of DoD medical facilities — judging what's feasible within DoD constraints and how each could inform UFC 4-510-01. New here? Take the 2-minute guided walkthrough, or explore on your own — full docs live under Help.

Sign in

AI Evaluation Matrix · UFC 4-510-01

Accounts are created by your administrator. Contact them if you need access.

☁ Cloud Sync

Connect a shared Supabase database so your whole team reads and writes the same evaluations. Leave it disconnected to keep data only on this device.

Not connected — data is stored only on this device.