In September, the Scottish Borders opened its first railway in 46 years – whisking passengers through a landscape of Roman ruins, medieval abbeys and even Robert the Bruce’s own heart.

I gazed down at the earth encasing Robert the Bruce's long-buried heart and wondered: might this count as the heart of Scotland? Spiritually, I meant. Geographically, I was on the country's southern edge, standing in the red-stoned ruins of Melrose Abbey, the 900-year-old icon of the area’s prettiest town.

The Scottish Borders, the rural region just northeast of the country’s boundary with England, couples distinctive beauty with an ancient past. Deep history is woven into the weft of its muscular heather-clad hills, lush woods and river-scoured valleys. The celebrated St Cuthbert's Way, which commemorates the 7th-century saint’s life, starts its 62-mile run to the holy English island of Lindisfarne here (though rugby is probably the bigger religion today, with Melrose hosting the world-renowned Rugby Sevens tournament each April). The area is home to medieval abbeys and Roman ruins, Sir Walter Scott’s home and a mining museum – not to mention, of course, to the heart of Scotland’s most famous warrior king.

Despite its draw, relatively few tourists tackle the Borders. That may be about to change. After nearly half a century as Britain's only region without a train, the area opened the Borders Railway in September. The 55-minute trip re-establishes a 30-mile route from Edinburgh to Tweedbank that was closed in Britain's 1960s cull of more than 5,000 miles of mostly rural track. I was among the first visitors to take the new railway into Scotland’s hauntingly beautiful, historic hinterland.

As we pulled out of Edinburgh’s Waverley station, the train cut through low-slung hills into a patchwork of woods and sheep-dotted slopes, criss-crossed by steel-grey streams. Just past the sixth stop of Gorebridge, the granite face of Borthwick Castle – now a five-star country hotel – towered over the tracks. At Stow, a gaggle of grinning kids pressed against a trackside fence, shrieking as we passed their school. But I was staying on to the end: the village of Tweedbank, about 25 miles north of the English border.

Tweedbank is a mile's walk west from Sir Walter Scott’s Abbotsford House, a slice of Edinburgh-style grandeur artfully set into lush gardens and woodland by the River Tweed. Scott's grand pile was stuffed with startling objects as colourful as his characters. A skull allegedly cast from that of Robert the Bruce gazed at me in the hallway. In a library whose brown leather-bound volumes suggested utter propriety, I discovered Scott's passion for witchcraft and demonology. Elsewhere, I found walrus tusks, the dagger of Scottish outlaw Rob Roy, and a silver urn that was a gift from Lord Byron, for whom Scott coined the immortal phrase “Mad, bad and dangerous to know”.

Tweedbank is the line’s terminus today. Although the long-term dream is to re-establish the whole of the 1840s train line to the English city of Carlisle, 60 miles to the south, engineering issues and rising costs have curtailed those plans for now. And so, from Tweedbank, I grabbed a taxi for the two-mile trip to neighbouring Melrose.

When Robert the Bruce’s heart was buried in Melrose in 1329, the town's abbey was already 200 years old. But the town’s connection with fierceness was nothing new. A thousand years earlier still, it was the Borders – not Hadrian’s Wall – that marked the true northernmost limit of Roman Empire in Britain, reaching up towards the Antonine Wall that once stretched across Scotland.

There are mementos of Roman settlement in the tiny Three Hills Roman Heritage Centre on Melrose's main square, which adds a Roman soldier's skull to a Scottish king's heart in the town’s collection of body parts. Christianity was just getting a foothold when the soldier died here, but within five centuries, early Borders churches laid the seeds for Britain's finest concentration of medieval abbeys. All are now in ruins, thanks to the 16th-century destruction wrought in the Scottish Reformation; Henry VIII personally led the bombardment of Kelso Abbey, just 15 miles east.

The Roman settlement of Trimontium, 1.5 miles east of Melrose, guarded a strategic spot on an ancient trail across the River Tweed, in the shadow of the Eildon Hills. Now a stone memorial and a few rocky outlines of an amphitheatre are all that remain. Still, guided walks to the site from Melrose (from April to October) are a grand excuse to admire the graceful arches of the Victorian railway viaduct at Leaderfoot and potter through Scotland's oldest continually inhabited settlement, the 2,000-year-old village of Newstead.

My next stop was a 30-minute, 13-mile bus ride south from Melrose to the town of Jedburgh, bowling past fields where countless rolled hay bales looked like golden toys scattered by a playful giant. The ruins of Jedburgh Abbey glowed in the afternoon sun. Around the corner was the Mary, Queen of Scots House, where the ill-fated queen resided in 1566. It was filled with sad mementoes of the tragedies that later enveloped her: a copy of her execution warrant signed by Elizabeth I; a shoe discarded when its heel broke; a death mask, the face set proud to the last.

Other ancient punishments were chronicled via a climb up from the town’s High Street to the Castle Jail, where exhibits of intriguing local history complemented the old cells. Jedburgh's impressive list of past high achievers made me want to plead guilty to the crime of ignorance: I read about the likes of astronomer John Veitch – who Sir Walter Scott called “one of the most extraordinary persons I ever knew” – and Mary Somerville, the 19th Century's “Queen of Science” and the woman after whom the Oxford college is named.

I learned about further generations of unsung heroes the next day at the National Mining Museum Scotland, housed in the old Lady Victoria Colliery atop a 1,600ft-deep mine by Newtongrange train station. A tour provided humbling insight into the mining communities that laboured to bring coal from the earth here since the 12th Century. My guide was Tom Young, a mining veteran of 14 pits; he spoke with quiet authority leavened with often blackly comic observations. “Before laying the new railway track, they bored holes every 20ft looking for old coal mine shafts to make sure your train wouldn't suddenly disappear into the earth... though they never did that with the old line!” he said.

Back above ground, I took the train back to Melrose and was gratefully to sucked in sweet, pure Scottish air as I walked down to the River Tweed, where a 19th-century chain bridge spanned the rushing waters just upstream of an old wooden fishing hut that was stood on stilts to protect it from seasonal floods. Back in Melrose, I popped into the idyllic, walled Priorwood Gardens, where bees buzzed through orchards laden with little-known medieval fruits like bullace (a type of plum).

Wandering again into the abbey’s grounds, I reflected one last time on the heart of Robert the Bruce. “A noble hart may have nane ease gif freedom failye” (“A noble heart can know no ease without freedom”), said the engraving on the stone cover that encased it. I was glad to have a new freedom – of movement, at least – in the Borders.

This story is a part of BBC Britain – a new series focused on exploring this extraordinary island, one story at a time. Readers outside of the UK can see every BBC Britain story by heading to the Britain homepage; you also can see our latest stories by following us on Facebook and Twitter.