People who are bad with numbers often find it harder to make ends meet – even if they are not poor