After reaching his first-ever half-century at stumps on the first day of play, de Swardt and von Berg collaborated for 70 runs off 27 overs in Hamilton. Von Berg was at 34 on his Test debut.
The tourists, having won the toss, made a comeback; they were 64-3 after lunch and 150-6 early in the last session after crucial batsman David Bedingham was freakishly removed.
An undermatched South Africa has to win at Seddon Park to avoid becoming the first team in their nation to lose a Test series against New Zealand, having lost the first Test match by 281 runs in Mount Maunganui.
During a tough 135-ball knock, De Swardt, one of six Proteas debutants in the first Test, demonstrated to the specialist batsman how to keep his wicket intact.
37-year-old leg-spinner von Berg, who got an international opportunity after playing 135 first-class matches, offered him support.
Both of them stopped a mini-collapse before the tea break, which was prompted by Rachin Ravindra, who scored 3-33.
Ravindra, who scored 240 in the first Test, underlined his all-round talent by removing Zubayr Hamza for a painstaking 20 off 99 balls and Keegan Petersen for two.
His left-arm spin then accounted for the unfortunate Bedingham, on 39, whose leg-side flick struck the top of his shoe and flew to close-in fielder Will Young.
South Africa made a dreadful start when makeshift opener Clyde Fortuin was dismissed for nought first ball in the second over, the wicketkeeper caught brilliantly in the gully by Glenn Phillips off Matt Henry.
Pace bowler Will O’Rourke claimed a scalp on debut when he had captain Neil Brand trapped leg before wicket for 25, while Raynard van Tonder was out for 32 in the over before lunch, caught at gully after failing to get on top of Neil Wagner’s short delivery.
South Africa boosted their spinning stocks, introducing von Berg and recalling off-spinner Dane Piedt for his first Test in four years.
They replaced opener Edward Moore and seamer Duanne Olivier.
For New Zealand, 22-year-old O’Rourke replaced Kyle Jamieson, who had back pain following the first Test.
Veteran seamer Wagner was recalled in place of spinner Mitchell Santner, while Young replaced batsman Daryl Mitchell, who was ruled out with a foot injury.
(With AFP inputs)