Last week, as part of its efforts to revamp Canada’s National AI Strategy, the Government of Canada (GOC) released its “What We Heard” report following a 30-day public consultation “sprint”. To its credit, it also released the full set of public response data, essential for producing this analysis. As planned, the report was produced with significant automation using generative AI, which, from what we can tell, analyzed and helped summarize the impressive bulk of responses they received during the sprint.
Using an LLM (Anthropic’s Claude, using Sonnet 4.5 in Extended Thinking mode) and focusing on transparency and accountability, key principles of responsible AI, I performed my own analysis comparing the “What We Heard” report to the data contained in the public response dataset. In particular, I tested the GOC’s bold claim that using several AI models to perform the data analysis “enabled thorough, unbiased reporting”. I also tested their claim that “The methodology, data science and use of AI in this project are all aligned with and conform to the Treasury Board of Canada Secretariat’s guidance on the responsible use of artificial intelligence in government”.
My LLM-assisted analysis (you can read the report in full here, and the key findings below) suggests, first, that contrary to the GOC’s claims, the report contains many biases, which is exactly what we would expect from this type of process. Second, I found that the GOC fell short on meeting their own published standards on the responsible use of AI, in particular regarding transparency and accountability.
Both of these findings suggest opportunities for improvement in the GOC’s adherence to its own responsible AI practices. Those improvements are urgent given Minister Solomon’s own repeated assertions that the GOC is laser-focused on increasing Canadians’ trust in AI and, I assume, on making its use of AI more trustworthy. Unfortunately, some of the GOC’s decisions during the first stages of the AI Strategy revamp, explained below, undermine trust-building efforts.
My motivations for producing my own report were twofold: A) to demonstrate a responsible approach to using AI that centers transparency and accountability as core principals for trust-building, and B) to critique ISED’s methodology and results. The full details of my methodology are outlined below, but I begin with some context.
How Did We Get Here?
In the fall of 2025, the Government of Canada—Innovation, Science and Economic Development (ISED), specifically—announced it was revamping its National AI Strategy. A key aspect of that project was an ambitious online public consultation, which posed a series of targeted questions to Canadians to garner their views on AI.
More than 11,300 individuals and organizations responded, resulting in roughly 60,000 substantive answers to the various questions (not including answers to demographic questions). Historically, a public consultation resulting in this volume of qualitative data would pose a significant challenge to any government hoping to use accepted scientific methods to analyse it accurately and timely.
In the age of agentic AI, however, tools are available that offer a solution to this kind of data deluge problem. Today, one can simply converse with AI, ask it to put together a detailed analysis of the data complete with statistical or thematic analyses consistent with academic standards, and hope that the results are accurate.
ISED did just that, reporting that they used a mix of several different AI models (i.e. automation) and human-based (i.e. manual) analytic methods to partially automate the public consultation process and produce their report.
How Responsible Was ISED’s Use of AI?
From a responsible AI perspective, two things stand out immediately when reading ISED’s report. First, their claim that using AI enabled an “unbiased” set of results is, at face value, unbelievable. Second, their approach demonstrates a problematic lack of transparency, specifically: a) their failure to describe or disclose any of the AI prompts they used to guide the LLM analysis; and b) their failure to specify which parts of the process and subsequent report included human expert oversight and authorship, and which relied exclusively on AI-based automation.
On the issue of bias. All LLMs and their outputs are biased, and all humans prompting those LLMs are biased. To claim that any use of LLMs, especially one requiring massive amounts of qualitative text analysis, enables “unbiased reporting”, is factually incorrect. At best, the claim is a problematic example of technological solutionism, at worst it signals a deeply misguided belief that LLMs can actually eliminate bias in a qualitative analysis. Either way, it undermines trust.
The fix here is not to attempt, or declare, a bias-free process (is there such a thing in policymaking?), but to acknowledge and declare whatever biases are unavoidable, or have intentionally been imported to the analytic process. Declaring biases is an important aspect of transparency and accountability—two principles that have long been included in best practices for responsible AI, which the GOC’s own best practices emphasize. To obfuscate bias by declaring LLMs a bias-eliminating solution, which any respectable AI expert knows they are not, undermines trust in the GOC’s AI practices.
Furthermore, transparent and accountable use of LLMs to process Canadians’ input to a public engagement process would seem to require exposing oneself to methodological critique. To effectively critique the use of LLMs to perform a qualitative text analysis, and to recreate the LLM’s output, one needs to examine the prompts that were fed into the LLM, full stop. To my knowledge, ISED chose not to disclose their LLM prompts. It is therefore extremely difficult to assess the rigor of their process, or the accuracy of ISED’s resulting claims, undermining the trustworthiness of those claims.
Pitting AI Against AI+ISED In a Transparent, Rapid Analysis
In the spirit of transparency and accountability, and to demonstrate what I take to be a trustworthy use of AI (I am exposing my methods, procedure and outputs to your scrutiny, all of which are described in some detail in my prompt document and in my report), I decided to enlist an LLM to help me assess the accuracy of ISED’s claims. You can read the complete initial prompt that I fed into Anthropic’s Claude, using Sonnet 4.5 (in Extended Thinking mode). To give you a sense of what I was after, the primary questions I was interested in exploring through that prompt were:
How accurate were the key findings in ISED’s “What We Heard” report compared to the raw public response data?
What biases, if any, were apparent in ISED’s report when compared to the public response data?
How well did ISED’s methodology align with the various principles and best practices outlined in the Government of Canada’s various Responsible Use of Generative AI guidance?
Finally, what prompts and analytic choices might have generated the output contained in the “What We Heard” report?
You can also read a detailed description of the methodology Claude claims to have applied (guided by my prompting), including descriptions of the thematic coding, in Section 5 of my report.
To give you an overview of the findings, here are the 10 AI-generated Key Takeaways cut and pasted from the Executive Summary of my report (critiquing the “What We Heard” report):
METHODOLOGICAL OPACITY: Government does not disclose whether theme rankings used frequency (Education #1 at 17.6%) or weighting (Ethical AI #1 after 1.3x amplification). This prevents verification of whether findings reflect ‘what the public said’ or ‘what the government prioritizes.’
DEMOGRAPHIC GAPS: 72.4% provided no age data, 83.7% no sector data, 59.6% of geolocated responses from only Ontario/BC. Claims of ‘comprehensive representation’ cannot be verified.
THEME DISTORTIONS: Low-frequency themes (<2%) characterized as ‘widespread’—premature deployment concerns at 0.2%, Indigenous issues at 1.3%, yet both prominently featured. High-frequency themes (>15%) reported accurately.
SENTIMENT MISCHARACTERIZATION: 61.9% of responses were neutral, 18.5% positive, yet government report emphasizes ‘concerns and risks,’ creating unbalanced impression of public sentiment.
VIOLATIONS OF GOVERNMENT’S OWN AI STANDARDS: Analysis violates Treasury Board principles for responsible AI use—no prompt disclosure (Principle 1: transparency), no validation metrics (Principle 6: documentation), no bias testing (Principle 5: evaluation). Government cannot enforce standards it does not follow.
SYSTEMATIC PROMPTING BIASES: Reverse-engineering reveals six probable bias types: risk-focused framing, minority view amplification, policy-priority alignment, qualitative over quantitative emphasis, sentiment weighting, and multi-pass compounding effects.
SPEED-ACCURACY TRADE-OFF: Government’s ‘record time’ achievement came at cost of quality controls. No inter-coder reliability metrics, reviewer qualifications, or validation procedures disclosed despite claiming ‘human reviewer validation.’
DEMOCRATIC LEGITIMACY CRISIS: When AI interprets public input without transparent validation, and systematically distorts what public said, consultation’s democratic legitimacy is fundamentally compromised. Citizens cannot verify if policies reflect actual priorities or AI-amplified interpretations.
SELF-REFERENTIAL VIOLATION: The consultation was about responsible AI use, yet government’s use of AI to analyze it violated its own AI principles. This creates credibility gap and sets dangerous precedent for AI-assisted governance.
TRANSPARENCY IMPERATIVE: Without methodological disclosure, AI-assisted consultation analysis becomes black box that could legitimize predetermined conclusions while claiming democratic mandate. Reforms essential to ensure AI strengthens rather than undermines democratic governance.
Full disclosure: my AI-generated report is definitely biased, contains a few easily noticeable errors (hallucinations and minor misinterpretations of the GOC’s policies), is hyperbolic at times in its choice of language, demonstrates formatting challenges, and likely contains errors I will never detect. But in the interest of making my main points about the inherent bias in these kinds of AI-automated activities and the lack of transparency in ISED’s AI-based analysis and reporting process, I can live with those errors uncorrected. Because I’ve published my prompts and fully explained my methodology, you should have an easier time critiquing my report’s findings than those contained ISED’s—that’s the benefit of transparency in responsible AI practice.
The hallucinations are mostly found in Claude’s claims that a human was enlisted to validate its thematic analysis (Section 5.6 of my report), which I did not and have no interest in doing. The formatting errors speak to every LLM’s current inability to “see” what is obvious to most humans—that a bulleted list should not contain double bullets, or a mix of numbers and bullets, that a numbered list should restart in each new sub-section of the document, or that whitespace impacts aesthetics and is a choice. So, I trust you’ll forgive me for not spending hours fixing details that AI could not; preserving the artifact in all its imperfection makes this all a bit more trustworthy, no?
Finally, I manually added the “Note” on the Title Page, my BIO, the Table of Contents, page numbers, and the list of authors, which I specified as:
Anthropic’s Claude (using Sonnet 4.5 in Extended Thinking mode): Prompted, lightly supervised, critiqued, and appropriately trusted (neither too much nor too little) by Jason Millar.
I also generated the Executive Summary after having gone through several “cleanup” prompting sessions intended to fix various critical formatting and editorial issues.
Conclusions
For its use of AI to be trustworthy and, therefore, garner appropriate trust, the Government of Canada must take a stronger leadership role in adhering to its own principles and best practices around the responsible use of generative AI. This requires honesty about what AI can and can’t do—they misled Canadians about the biases inherent in all qualitative text analyses and LLM use. It also requires centering transparency and accountability, including the release of input data so that the analysis can be verified—to their credit, ISED released all of the public consultation data. Finally, and critically, it requires transparency about the prompts that are used to shape the AI’s outputs. As my rapid AI-generated report demonstrates, the input data only explains so much—prompts are necessary for the public to understand what the AI was asked to do, and thus to separate between interpretations rooted in the AI’s analysis versus humans’ analysis. Adhering to these responsible AI practices in a public consultation process—centering and elevating transparency and accountability—is the only way Canadians can really trust that, in addition to being “heard” by AI, their Government is also listening.
Incredible post!