This is a interesting topic perhaps has an potential to connect to the roots of the Sanatana Dharma. It has always been used as a traditional bait by critics of Sanatana Dharma who would like to advocate Sanatana Dharma to be a faith based on mythological fables and nothing more than that.
However they generally tend to neglect any arguement that would give any crediblity to Sanatana Dharma. That has always been the stand of majority of the western Indologist who would always like maintain their European racial superiority over Indians. However traditional scholars from the sub-continent never had any problem because they always had the scriptural perspective on “mythological” stories. Srila Prabhupada one such scholar here states in his commentary on Srimad Bhagavatam the reason why the earthly beings are unable to see the celestial beings. They are beings of higher dimension and are superior to humans in all respects, in fact they have ascended to heavenly dwelling on account of thier piety they have accumulated in the Earth. So they are normally dis-interested in the affairs of the people in this dimension. But they do visit this dimension when there are acts of piety performed by certain great souls like Maharaja Parikshit. More in the Purport from Srila Prabhupada.
TRANSLATION: Mahārāja Parīkṣit, after having selected Kṛpācārya for guidance as his spiritual master, performed three horse sacrifices on the banks of the Ganges. These were executed with sufficient rewards for the attendants. And at these sacrifices, even the common man could see demigods.
PURPORT: It appears from this verse that interplanetary travel by the denizens of higher planets is easy. In many statements in Bhāgavatam, we have observed that the demigods from heaven used to visit this earth to attend sacrifices performed by influential kings and emperors. Herein also we find that during the time of the horse sacrifice ceremony of Mahārāja Parīkṣit, the demigods from other planets were visible even to the common man, due to the sacrificial ceremony. The demigods are not generally visible to common men, as the Lord is not visible. But as the Lord, by His causeless mercy, descends to be visible to the common man, similarly the demigods also become visible to the common man by their own grace. Although celestial beings are not visible to the naked eyes of the inhabitants of this earth, it was due to the influence of Mahārāja Parīkṣit that the demigods also agreed to be visible.
Also Srila prabhupad would also offer a counter arguement that we can’t see the Prime Minister of India when we think of meeting him. It requires lot of efforts and influence to get any closer to Prime Minister. So what to talk of the Supreme person he is not our servant to be present before us when we demand. It required 5 year old Dhruva to perform unbroken austerities to extent of abstaining from even water for six whole months in madhuvana to get the audience of the Supreme Person. Hiranyakasipu performed severe austerities for thousands of years to see Lord Brahma. That is the way to have audience of the Supreme Lord.