Irish outfit Ulster required two converted tries in the last 10 minutes to edge ahead of a willing Southern Kings team.
Ulster eventually ran out 43-36 winners in a 12-try thriller in the historic match - played at the Wolfson Stadium in KwazaKhele, a township of the city of Port Elizabeth.
The Kings held the lead for most of the match and it was not until the 75th minute that Ulster finally grabbed the lead for the first time.
They had to hang on desperately, as the Kings launched a desperate bid to draw level and give themselves a chance of getting their first Pro14 win.
The Kings raced out of the blocks and were 22-10 up at the break, but a volley of late scores at both ends ultimately ensured Ulster came away with a bonus-point victory.
The home side started the game at breakneck speed and was over the line twice within the first 10 minutes.
Berton Klaasen dived over the whitewash within four minutes to give the home side the lead, which was doubled almost instantly when fullback Yaw Penxe charged through after picking up a grubber.
After missing his first attempt in very tricky conditions, Oliver Zono slotted the conversion to pull the hosts ahead 12-0 and leave Ulster shell-shocked.
However, the away side got the first points of the day on the board soon after in style through No. 8 Sean Reidy, who applied the finishing touch to a sweeping Ulster attack to make it 12-5.
John Andrew crashed over from a rolling maul minutes later as the visitors grew into the game.
They should have been in the lead soon after following a tremendous break from centre Darren Cave, who was impressive all afternoon.
But in a play that typified the ding-dong nature of a thrilling match, Kings winger Michael Makase intercepted Cave's pass to the left and kicked up field before applying the afterburners.
Makase made up sensational ground to beat Charles Piutau to the loose ball and grounded with a desperate lunge to put the hosts back into the lead, Zono adding the extras.
The second half was even more breathless than the first, with Les Kiss' side striking two decisive blows early on, first when Andrew crossed for his second after another effective rolling maul, and John Cooney dotting down under the posts to level proceedings at 22-22 and net Ulster a bonus point.
The Kings, winless and without a single point in Conference B before today's contest, claimed a bonus point of their own through Klaasen's converted score, sparking a frenzy of activity near both try lines until the culmination of the match.
And back came Ulster, with Andrew Trimble ghosting through after clever work by Cooney for his 50th league try, with Christian Leali'ifano doing the honours from the tee.
Clive Ross charged over with ten minutes to go as Ulster took the lead for the first time in the match, but Penxe hit back instantly for the Kings as proceedings were squared once more with moments left.
Both sides had shown adventure and commitment all afternoon, but Ulster's class came through in the end, Diack forcing his way over in the right following smart build-up play from Piutau for the game's 12th and final try.
For the Southern Kings:
Tries: Klaasen 2, Penxe 2, Makase
Cons: Zono 4
Tries: Reidy, Andrew 2, Cooney, Trimble, Ross, Diack
Cons: Cooney 2, Lealiifano 2
Yellow card: Yaw Penxe (Southern Kings, 33 - cynical foul, slapping the ball down)
Southern Kings: 15 Yaw Penxe, 14 Michael Makase, 13 Berton Klaasen, 12 Luzuko Vulindlu, 11 Alshaun Bock, 10 Oliver Zono, 9 Rudi van Rooyen, 8 Ruaan Lerm, 7 Jurie van Vuuren, 6 Andisa Ntsila, 5 Bobby de Wee, 4 Stephan Greeff, 3 Rossouw de Klerk, 2 Stephan Coetzee, 1 Schalk Ferreira (captain).
Replacements: 16 Tango Balekile, 17 Mzamo Majola, 18 Martin Dreyer, 19 Lindokuhle Welemu, 20 Dries van Schalkwyk, 21 Rowan Gouws, 22 Kurt Coleman, 23 Jarryd Sage.
Ulster: 15 Charles Piutau, 14 Craig Gilroy, 13 Louis Ludik, 12 Darren Cave, 11 Andrew Trimble, 10 Christian Lealiifano, 9 John Cooney, 8 Sean Reidy, 7 Chris Henry (captain), 6 Matty Rea, 5 Alan O'Connor, 4 Kieran Treadwell, 3 Wiehahn Herbst, 2 John Andrew, 1 Andy Warwick.
Replacements: 16 Adam McBurney, 17 Callum Black, 18 Ross Kane, 19 Robbie Diack, 20 Clive Ross, 21 Jonny Stewart, 22 Peter Nelson, 23 Tommy Bowe.
Referee: Quinton Immelman (South Africa)
Assistant referees: Rasta Rasivhenge, Ben Crouse (South Africa)
TMO: Shaun Veldsman (South Africa)