On 27 November 2025, the Delhi High Court directed GST and Income Tax authorities to exercise “utmost caution” and assume full responsibility whenever judgments cited in official communications are generated or suggested by Artificial Intelligence (AI) tools.
Background of the Case: AI Hallucinations in GST & Income Tax Notices
In JSK Industries Ltd. v. Union of India & Ors. (W.P.(C) [2025]), the petitioner challenged a show-cause notice (SCN) issued under Section 74 of the CGST Act, 2017, alleging tax evasion exceeding ₹88 crore through suppressed sales, maintenance of parallel books of accounts, and clandestine use of a “JSK Server”.
The SCN originated from intelligence gathered during an Income Tax search. On 27 November 2025, a Division Bench comprising Justice Prathiba M. Singh and Justice Shail Jain dismissed the writ petition as premature, directing the petitioner to file a reply and actively participate in the adjudication proceedings.
Judicial Observations on the Use of AI in Tax Administration
While disposing of the petition, the Bench made far-reaching observations on the increasing reliance on AI tools:
- The Court physically retrieved law reports from its library and verified the judgments cited in the SCN. It found that two of the citations were either non-existent or materially incorrect — a classic instance of AI hallucination.
- Referring to its own earlier landmark ruling in Christian Louboutin SAS & Anr. v. M/S The Shoe Boutique (2023), the Bench reiterated:
“Accuracy and reliability of AI-generated data is still in the grey area… At the present stage of technological development, AI cannot substitute either the human intelligence or the humane element in the adjudicatory process. At best, the tool could be utilised for a preliminary understanding or for preliminary research and nothing more.”
The Court issued a clear directive:
“The GST Department as well as the IT Department must exercise utmost caution while citing judgements and must take full responsibility in case the same is cited or generated by using Artificial Intelligence software… Before issuing SCNs or finalising assessments, all judgements ought to be verified.”
The Growing Risk of AI Hallucinations in Legal Proceedings
The judgment highlights the well-documented danger of Large Language Models fabricating non-existent case laws — commonly termed “AI hallucination”. Indian courts, including the Supreme Court, have repeatedly criticised pleadings containing fictitious citations produced by ChatGPT and similar tools. Globally, several U.S. courts have imposed sanctions and fines on lawyers for citing AI-generated fake precedents.
Legitimate & Responsible Uses of AI in Tax Administration
Despite these risks, the Court implicitly recognises that AI can play a valuable supportive role when subject to rigorous human oversight:
- Data analysis & anomaly detection – processing vast transactional data to flag potential evasion.
- Preliminary legal research – quickly locating statutes, circulars, and authentic judgments.
- Drafting assistance – generating structured summaries and initial drafts of notices.
- Predictive analytics – forecasting revenue trends and identifying high-risk taxpayers.
- Operational efficiency – automating repetitive tasks, freeing officers for complex adjudication.
Critical Caveat: Every AI-generated output, especially legal citations, must undergo independent human verification before incorporation into any judicial or quasi-judicial document.
Key Takeaways for Tax Authorities and Practitioners
- AI may be used only for preliminary research or drafting support.
- All citations in SCNs, draft orders, or assessment orders must be independently verified — preferably through physical law reports or authenticated databases.
- Departments bear full responsibility for any erroneous or fabricated judgments introduced via AI.
- Physical verification of cited judgments is now effectively mandatory before issuing notices or final orders.
- The ruling reinforces the broader judicial consensus that human oversight remains indispensable in adjudication.
Conclusion
The Delhi High Court’s directive is a timely wake-up call. While artificial intelligence offers powerful tools to modernise tax administration, it remains an assistive technology — not a replacement for human diligence, accountability, and legal acumen.
As AI adoption accelerates across CBIC and CBDT, robust verification protocols and continuous training of officers will be essential to harness innovation without compromising the integrity of quasi-judicial processes.
Case Citation: JSK Industries Ltd. v. Union of India & Ors., W.P.(C) [2025] (Delhi High Court, decided on 27.11.2025)
What are your views on the use of AI in GST and Income Tax proceedings? Share your thoughts in the comments below.