Processes for dealing with rape and other serious sexual crimes in NI take too long and fail to deliver an acceptable outcome for victims.
Criminal Justice Inspection says the system "frequently does not provide a satisfactory outcome for victims on any level".
The number of sexual offences reported to police has almost tripled over the past 18 years.
The Public Prosecution Service said it accepted the report's recommendations.
There is now an average of nine reports of sexual offences per day.
But the conviction rate in rape cases in Northern Ireland is under 2%, lower than other parts of the United Kingdom.
The inspectors say victims are being let down by the police and Public Prosecution Service (PPS).
Their findings are included in a report published on Tuesday.
"This report concludes that the criminal justice processes in Northern Ireland for handling these cases take too long, are too expensive and conclude with, all too often, a failure to deliver an acceptable outcome for victims," said chief inspector of criminal justice Brendan McGuigan.
"Victims, who may have waited months, years or decades to report the offences against them to the police, are let down by a system which then takes months or years for an investigation, and prosecution to commence and years to have the case resolved in court."
Geraldine Hanna, from Victim Support NI, said while the capacity to cope with the volume of cases was not there, this was "not an excuse".
"Delays in processing cases is often given as a reason as to why victims of sexual offences withdraw from the system," she added.
"Also, the particular nature of sexual violence cases means there is often limited, if any, corroborating evidence.
"Memory therefore is vital. Delay impacts on the victim's ability to recall and therefore can make it more difficult to recount the truth."
The report says more than 3,150 sexual violence and abuse offences such as rape, sexual assault and child abuse were recorded by police during the year 2016-17.
Of these, 823 were offences of rape, with 15 of the defendants convicted of rape when the case went to court - a conviction rate of 1.8%.
In comparison, the proportion of rape cases resulting in a conviction in England and Wales during the same period was 3.3%, while the rate in Scotland was 3.6%.
"Each one of these incidents represents a significant trauma for the victim and creates an expectation that for those who have chosen to report the offence, and seek a criminal justice outcome, their case will be progressed in a timely fashion," Mr McGuigan says.
"This report however has shown that this is unlikely to be the case.
"Inspectors found that while there were many dedicated and professional individuals involved in dealing with incidents of sexual violence and abuse, delay was a substantial problem.
"It occurred from when the initial police investigation and file preparation was undertaken, through the time taken by prosecutors to review the case and make a decision to prosecute or not, and afterwards in the number of adjournments at court before a trial commenced."
'Myths and and stereotypes'
The report makes nine recommendations for improvement, including a radical change to the way serious sexual cases are processed by the courts.
Currently, the cases are subject to a committal hearing to decide whether there is sufficient evidence for a defendant to be tried in a Crown Court.
The inspectors say the current system "causes delay, prolongs the trauma on victims and potentially leads to their withdrawal from the case".
They believe proposed changes for other indictable offences such as murder and manslaughter should also apply to serious sexual crimes.
These offences are ones where a defendant has the right to be tried by a jury, and which may attract a punishment of imprisonment of more than a year to life.
The proposed changes will mean murder and manslaughter cases will go straight to Crown Court.
The inspectors say adapting a similar approach for serious sexual crimes such as rape would "reduce the anxiety for victims and should reduce delays in case progression".
They also call for improvements in the handling of "myths and stereotypes" in cases involving sexual offences.
These would include any suggestion in court that the way an alleged victim was dressed, or the fact that they were drunk, may signal consent.
One of the recommendations is that the Department of Justice should introduce legislation containing a requirement for a judge to give a jury directions "to enable them to approach court evidence in a more informed way".
'Rigour of the system'
Marianne O'Kane, the assistant director of the Public Prosecution Service, said the PPS is "collaborating with our criminal justice partners on a shared action plan".
"We very much understand that the rigour and robustness of the system can be difficult from a victim's perspective, and this is particularly true of those who are a victim of a sexual crime," she said.
"For our part, we are wholly committed to working with our criminal justice partners to improve how we deliver together, with fairness and integrity.
"Work is already well under way to deliver the agreed PPS actions."