Since getting permanent residence, have you spent more than 90 days outside South Africa in any one year?
The regulations say an applicant absent more than 90 days in any year of the qualifying period 'does not qualify for naturalisation'. It is a flat bar with no allowance for the reason. Count from the stamps in all your passports, year by year.