Transgender people in the UK face "high levels of transphobia" on a daily basis and they have "a long way to go" to achieve equality in the UK, MPs say.
The Commons Women and Equalities Committee urged ministers to draw up a new strategy to tackle discrimination.
The NHS and prison service need urgent reform, its MPs said, and there should be more training for police officers and teachers in handling trans issues.
Minister Nicky Morgan said further progress on equality was a priority.
The minister for women and equalities said she was determined to tackle discrimination wherever it existed.
It estimates that as many as 650,000 people in the UK are "gender incongruent to some degree" and the transphobia they experience undermines their careers, incomes, living standards and mental and physical health.
About a third of transgender adults and half of "gender-variant" young people attempt suicide, it says.
Among more than 30 recommendations, the report calls for:
- A "root-and-branch review" of the NHS's treatment of transgender people.
- Urgent clarification on the position of transgender prisoners, given the "clear risk of harm" if trans people are held in prisons according to their birth genders. It cites the cases of two transgender women who died in 2015 while serving time in male jails.
- Official recognition of gender should be based on "self-declaration", rather than a "medicalised" assessment.
- Mandatory training for police officers on transphobic hate crimes, and the extension of hate crime laws to cover gender identity.
- The lowering from 18 to 16 of the age limit for obtaining official recognition of a new gender without parental consent.
- The option to record gender as 'X' in a passport, and an end to the need to show a doctor's letter to alter the gender shown.
- More training for school staff to better support "gender-variant" young people.
- Guidance for sports bodies to make clear that exclusion of transgender players on grounds of safety or fair competition is rarely justified.
'We have to prove our identity'
"We have a gender-based care system which pathologises trans people," says Tara Stone, chair of Be: Trans Support and Development North, based in Newcastle.
"Currently, if a trans person goes to their GP for help, the procedure is to eliminate every other possible thing that could be wrong and what you're left with is, 'OK, we believe you.' We need to move away from that and just start with, 'Right, we believe you.'
"We have to prove our identity - nobody else in society has to do that - and it leaves people with a feeling of shame, like there's something wrong with them.
"Recently a girl I've worked with went to her GP for a referral, but the response was, 'I think you need to go away and think about it. Come back in a fortnight's time.' She's entitled to gender identity services, but she was denied that.
"At the point of seeking help from a clinician, a lot of trans people are very vulnerable and, even with the best of intentions, that kind of response risks having a massive impact on their developmental path as a person.
"The trans community in general has a lot of distrust of institutions, because they feel they have been institutionally abused - by the NHS, the police, the government. They then resist engaging with those institutions, meaning they become even more isolated."
The report singles out the NHS, in particular, for criticism, warning of "significant problems... due to the attitude of some clinicians and other staff", driven by "lack of knowledge and understanding and even in some cases, out-and-out prejudice".
There is "overwhelming" evidence of "serious deficiencies in the quality and capacity of NHS gender identity services", it says, and "completely unacceptable" waiting times for first appointments and surgery.
Puberty-blocking drugs, which delay the development of adult sexual characteristics, and cross-sex hormone treatments should be provided more readily to young people, given the danger of self-harm and suicide if treatment is withheld, the MPs add.
However, the committee stops short of backing calls for official recognition of changed gender for children aged under 16.
Tara Stone works with All About Trans, a not-for-profit project that promotes trans voices in the media, encourages better understanding of trans people and improves media coverage of the issues trans people may face.