The deeper half of the ocean did not get measurably warmer in the last decade, but surface layers have been warming faster than we thought since the 1970s, two new studies suggest.
Because the sea absorbs 90% of the heat caused by human activity, its warmth is a central concern in climate science.
The new work suggests that shallow layers bear the brunt of ocean warming.
Scientists compared temperature data, satellite measurements of sea level, and results from climate models.
Both the papers appear in the journal Nature Climate Change.
Specifically in the Southern Hemisphere where fewer measurements have been made, a team of researchers from the Lawrence Livermore National Laboratory in California investigated long-term warming in the top 700m of the ocean.
They wanted to compare published warming rates, based on the sparse temperature data recorded directly from southern waters, with what could be predicted based on more detailed observations in the north, together with climate models and precise sea-level measurements.
Sea levels across the planet are accurately assessed by satellites, which bounce radio waves off the ocean surface. And sea level changes are closely related to ocean temperature, because the water expands as it warms up.
By combining these calculations, the scientists found that the rate of upper-ocean warming between 1970 and 2004 had been seriously underestimated. That inaccuracy is specific to the Southern Hemisphere, but is big enough, the scientists suggest, that global upper-ocean warming rates are also "biased low" - to the tune of 24% to 55%.
The researchers say the underestimation probably arose simply from the scarcity of measurements in the south.
"It's likely that due to the poor observational coverage, we just haven't been able to say definitively what the long-term rate of Southern Hemisphere ocean warming has been," said lead author Dr Paul Durack.
"It's a really pressing problem - we're trying as hard as we can, as scientists, to provide the best information from the limited observations we have."
Dr Jan Zika, an oceanographer at the University of Southampton, agreed that Dr Durack's work highlighted the need for more data.
"It could be that the heat uptake by the ocean is a stronger contribution than we previously thought," he told BBC News.
Storage of heat in the oceans is the best explanation scientists have for the recent "'slowdown'" in the rise of global surface temperatures.
Dr Zika added that the heat content of shallow layers is a particular concern, as that warmth is more likely to return to the atmosphere. "If it were getting really deep down, then we might think that it's basically stuck there," he said.
Meanwhile, at depths greater than 2km, temperatures appear relatively stable - at least as far as changes can be detected.
These devices started to be deployed in 1999 and there are now more than 3,600 of them adrift in our planet's oceans.
Argo "floats" only record temperatures down to about 2km beneath the surface, so the Caltech researchers used that information to calculate the overall amount of heat absorbed by the upper 2km of the oceans (approximately the top half).
Then, by subtracting that quantity of heat from the total ocean warming indicated by satellite observations of sea level, they estimated how much the deeper half of the ocean (below 2km) had warmed up.
The result: once the high level of uncertainty associated with the calculations was taken into account, there was no change detected in the deep.
"Combining those different observations, we ended up with a large uncertainty in our estimate," explained Dr William Llovel, the study's lead author.
"But on the other hand, it is a global estimate. We're not using just a few observations from the deep ocean itself."
In a commentary for Nature Climate Change, Dr Gregory Johnson and Dr John Lyman, both from the National Oceanic and Atmospheric Administration in Seattle, US, said the results pointed to a crucial gap in ocean measurements.
"It is time to close the deep-ocean measurement gap and reduce the uncertainties in global planetary energy and sea-level budgets," wrote Dr Johnson and Dr Lyman.
Dr Zika agrees. "There are very clear, very plausible explanations for how the ocean could be taking up heat and offsetting global warming," he told the BBC.
"But to attribute exactly where that heat's going, and whether it'll pop back out or not, is going to take more observations. We need to get those in place."
Follow Jonathan on Twitter