A former inspector at the Care Quality Commission says a 2015 report into Whorlton Hall hospital which presented "warning bells" went unpublished.
Barry Stanley-Wilkinson says he wrote the report four years before BBC Panorama revealed the alleged abuse of patients with learning disabilities and autism.
The CQC said the draft report raised no concerns about abusive practices.
The claims come after 10 workers at the specialist hospital were arrested.
Seven men and three women were arrested last week at addresses in Barnard Castle, Bishop Auckland, Darlington and Stockton over the alleged abuse of patients.
An undercover BBC Panorama investigation into the specialist hospital in County Durham - a 17-bed unit for adults with learning difficulties and autism - appeared to show patients being mocked, intimidated and restrained.
Cygnet, the firm that runs the 17-bed hospital unit for adults with learning difficulties and autism, said it was "shocked and deeply saddened" by the allegations.
The company only took over the running of the centre at the turn of the year and said it was "co-operating fully" with the police investigation.
The site had at least 100 visits by official agencies in the year before the alleged abuse was discovered.
Mr Stanley-Wilkinson says he noticed a "very poor culture" was evident when he led the 2015 inspection.
He told the BBC that he had raised concerns over the "very poor culture" in a report he wrote - four years prior to the BBC investigation.
He said: "I strongly believe that anybody that can understand organisational culture reading that report would agree that there was definitely warning bells there.
"I was extremely upset. This should have been listened to back in 2015 and I said quite openly, when I left the organisation, that I felt it had neglected its promise to people with learning disabilities."
He said it was the only report he wrote in nearly a decade of working at the CQC which wasn't published.
In a statement, the CQC said the report went through a "rigorous peer review process".
It said the draft report "did not raise any concerns about abusive practice".
The CQC said a later inspection rated the hospital as "good overall".
In a statement it said: "We are in the process of commissioning a review into what we could have done differently or better in our regulation of Whorlton Hall and these allegations will be fully investigated as part of this.
"We will update on the progress and findings of this review in our Public Board meetings."