Making Technical Readiness Assessment Work

The commonest failure in a technical readiness assessment is a scope defined by the technology when the decision at stake concerns the job the technology has to do. The delivery team lists its output, explains the test environment, and requests a reviewer to verify the gap between the two. That arrangement is honest, but it still produces a report no one can use, because the assessor never received the question the business actually needed answered.

What follows is a checklist. Readiness work fails at predictable stages, and each stage has its own questions. Answer each item below against your own programme with a yes or a no. A no marks a decision still open, and these open decisions are where the value of the exercise lies.

Perform these checks before you commission a project

You can set the decision the assessment informs and the deadline by which it must be made. If that part of the text is difficult to compose, the report will be filed and the programme will continue on the timetable it always had.

The signatory for the readiness finding has a role separate from the team delivering the technology. Self-assessment is legitimate for internal tracking and ceases to be adequate when the result will release funding, support a procurement or satisfy a regulator.

The assessor's fee is the same whatever the outcome. Independence is a structural feature of the arrangement, and it makes the finding defensible to a reader who was absent from the sessions.

You have confirmed that a readiness assessment is the correct tool for the decision you face. An ISO 27001 readiness assessment, as CyberOne sets out in its UK guide, is a pre-audit diagnostic measured against a fixed control set, and it surfaces non-conformities before an accredited certification body reaches them. Where the question concerns whether a technology can perform a specific job in a specific environment, that diagnostic was built for a different purpose, and the material it produces speaks to audit readiness.

The scope is fixed: The scope includes what is covered, what is excluded, and who made that decision. A scope that can be widened during the work will be widened whenever a finding looks awkward.

Checks on the evidence rules, before any score is given

Definitions are settled before data collection begins. The TRL-IS study published in Heliyon had to produce a standard definition of readiness first, because the literature it reviewed offered several incompatible ones, and rating maturity against an imprecise term yields values that are not comparable.

You know what counts as evidence, and who is qualified to offer it. The Technology Readiness Assessment Guide from the US Government Accountability Office presents a high-quality assessment as one with credibility, objectivity, reliability and usefulness, and it says that relatively few federal agencies had guides of their own before this one appeared. The same guide says that technologies less mature than recommended have driven programme delays and cost increases in major acquisitions. Those four properties describe the process behind the report, and a reader can test each of them whatever the document looks like.

The scoring scale is fixed before the first item is rated, and each level represents a recognisable feature in your environment. Scales imported from another sector carry that sector's assumptions with them, which is why the TRL-IS adaptation removed laboratory testing from the ladder and sharpened the line between a pilot in a relevant setting and a demonstration in the real one.

Multiple individuals, or the same individual at two different times, produce ratings so that disagreement surfaces as disagreement. Inter-rater reliability is measurable, and the TRL-IS authors measured it, reporting good agreement between raters of differing experience.

Checks while the assessment is running

Interviews include the people who operate the technology as well as the people who designed it. A readiness position assembled entirely from the design team's account tests the design and leaves the operation untested.

Evidence comes from the environment where the system will run. Testing under controlled conditions establishes that something works when the conditions are controlled. Real data volumes, real users and real failure modes belong in the evidence only once they have been exercised.

Manufacturing and supply questions are in scope wherever the technology will be produced at volume. The Manufacturing Technology Centre built its Technology Readiness Measurement Toolkit with Heriot-Watt University because levels inherited from aerospace and defence fit early-stage companies poorly, and it folded supply chain readiness and design for manufacture into the measurement itself.

Somebody outside the programme can read the evidence trail. If following a finding back to its source requires the original author to be present and explaining, the assessment depends on one individual's memory, and the documentation must remain valid once they have left.

Checks on the report before you accept it

Findings are ranked by the decision they affect, on an explicit basis. One workable basis is the one used in the TOGAF business transformation readiness case study, where each readiness factor was rated for urgency, readiness status and the degree of difficulty involved in fixing it. Ranking on a single axis hides the items that matter most.

Assessment is separated from remediation. The WireApps technical readiness report provides a prioritised set of findings, and the work of acting on them is scoped as a separate engagement. That separation keeps the assessment honest, because a provider expecting to sell the fix has an interest in how the problem is characterised.

Every gap has an assigned owner and a metric to indicate if it has closed. If one of those two is missing, the item remains in the report as prose and never reaches a plan.

The readiness tools differ

Readiness assessments are often compared as though they were variants of one method. They answer different questions, rest on different evidence and tolerate different degrees of independence.

Instrument	Question it answers	Evidence base	Independence expected	Lapses
Technology readiness assessment	Can this technology perform the required job where it will actually run?	Test and demonstration records, operator accounts	Assessor drawn from outside the delivery team	No, though the underlying position shifts as the environment shifts
Certification readiness diagnostic	Will this management system satisfy a fixed standard at audit?	Document review, control sampling	Assessor separate from the certification body	Yes, controls drift after the certificate is issued
Business transformation readiness	Can the organisation absorb the change the technology implies?	Facilitated workshops, maturity ratings against named factors	Facilitated, with accountable owners present	No, though factors re-rate as the organisation changes
Supplier readiness product	What is the current level of maturity, and what would raise it?	Questionnaire and interviews	Commercial provider	Depends whether ongoing work is bundled with the assessment

Comparisons based on the GAO's Technology Readiness Assessment Guide use details provided by CyberOne, Visual Paradigm and WireApps regarding their own tools.

The supplier market reflects that spread. The AI Readiness Assessment listed by BedigitalUK on the Digital Marketplace covers AI literacy, talent, management support, culture, governance, risk, security, business value and data readiness, and the work is priced by the day. That shape is informative: A questionnaire-led product provides a quick review of position across several defined areas, and it does not determine whether one system performs one job. A team that wants to see the questions before paying for an engagement can work through a free AI readiness assessment, and the categories alone will show whether the tool addresses the decision they have recorded.

The test to run on any finding you are handed

One test applies to any tool you use. Request that the assessor show the evidence for one finding and the day that evidence was created. The assessment is doing its job if the trail leads to a specific test, a specific environment and an expert who can be questioned about it. If it leads to a summary paragraph written late in the process, you have bought a document.

The position an assessment defines changes as the environment around it changes. A finding issued against one configuration, one load or one supply chain relates to that state at that time, and the report reads as current only for as long as the reader remembers the time on it.