I didn't realize but he was deeply unhappy at that time.

我没有察觉到原来他那时非常不开心。